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ABSTRACT 

This study used an iterative Delphi survey process of 
about 600 faculty, employers, and policymakers to identify writing, 
speech and listening, and critical thinking skills that college 
graduates should achieve to become effective employees and citizens 
(National Education Goal 6). Participants reached a consensus about 
the importance in critical thinking of the ability to detect: 
indirect persuasion including the use of leading questions that are 
biased towards eliciting a preferred response, use of misleading 
language, use of slanted definitions or comparisons , and instances 
where irrelevant topics or considerations are brought into an 
argument to divert attention from the original issue. With regard to 
effective writing respondents thought that graduates should be able 
to use active or passive voice where appropriate, use correct 
grammar, use specific language conventions of their academic 
discipline, and U5e language that their audience understands. With 
regard to speech communication skills respondents reached agreement 
abou^ the importance of inf ormat i on exchange , conversation 
management, group communication, and using and understanding spoken 
English and non-verbal signs. Extensive tables detail the Delphi 
survey results. (Contains 168 ref er*: nces . ) (JB) 
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FOREWORD 



National Education Goal 6 (originally listed as Goal 5) calls for the enhancennent of college 
graduates' ability to think critically, comnnunicate effectively, and solve problems. This 
working paper was connmissioned by the National Center for Educational Statistics (NCES). It 
is the fifth report outlining NCES planning activities to develop a process for the assessment of 
college student learning. The purpose of this study was first to identify, and then try to reach a 
consensus, among a group of faculty, employers, and policymakers, on the specific higher 
order communication and thinking skills that college graduates should achieve to become 
effective employees in the workplace and citizens in society. Although the 600 "judges'' 
consulted v^ere not necessarily representative of the universe of faculty, employers, and 
policymakers, the working paper does expand the number of individuals involved in making 
these expert judgments. In addition to reporting on the results of a two stage iterative Delphi 
survey process, the report highlights panicipants comments that provide insight why some 
agreed or disagreed about the importance of specific skills. It is to be noted that the working 
paper was not intended to provide a prescribed set of communication and critical thinking 
skills, but rather to stimulate discussion on the teaching/learning of these skills among 
educators. The ultimate choice as to which of these skills should be taught and mastered by 
college graduates will rest with each institution and its faculty. A future publication will provide 
^unilar information on problem solving skills with a working paper devoted to the reading 
skills expected of college graduates to follow. 

This work was conducted by the National Center on Postsecondary Teaching, Learning, and 
Assessment of the Pennsylvania State University, James Ratcliff, Director. It was funded under 
Department of Education Contract No. R 1 1 7G 10037. The project team was directed by 
Elizabeth A. Jones. She was assisted by Steven Hoffman, Lynn Melander Moore, Gary 
Ratcliff, Stacy Tibbetts, and Benjamin A. L. Click, III. For more information on the project, 
and earlier publications, contact Sal Corrallo, NCES Project Planning Director, 555 New Jersey 
Avenue NW, Washington, D.C. 20208, (202) 219-1913 (Voice) or (202) 2 1 9- 1 80 1 (FAX). ' 

Emerson J. Elliott 

Commissioner of Education Statistics 
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I. BACKGROUND AND PURPOSE 



In Winter 1990, the President of the United States and state governors announced six 
educational goals for the nation that are to be achieved by the year 2000. Goal Five states 
"every adult American will be literate and will possess the knowledge and skills necessary to 
compete in a global economy and exercise the rights and responsibilities of citizenship;' 
Specifically, Objective Five of Goal Five recommends that ''the proportion of college 
graduates who demonstrate an advanced ability to think critically, communicate effectively, 
and solve problems will increase substantially.'' What constitutes advanced abilities in these 
areas, and how do we know if students have mastered these skills? The National Center for 
Education Statistics (NCES) has been seeking an effective means for monitoring progress and 
measuring these advanced intellectual skills toward the auainment of this goal. 

Such an assessment must have validity among the colleges and universities where the students 
are assessed in order for improvement to occur. Criteria and measures of writing, speech and 
listening, and critical thinking skills need to have validity with national experts in those three 
areas as well as faculty teaching those skills at the wide variety of institutions where students 
may choose to complete their degree programs. 

Employers, policymakers, parents, and the public have been concenied with the achievements 
of college graduates. Employers are troubled by deficiencies in skills among new workers. The 
inability of large numbers of new employees to meet the reading, writing, or computational 
standai'ds required by many segments of American business is an economic and competitive 
issue for United States companies challenged by foreign enterprises (Carnevale, Gainer, and 
Melzer, 1990). As a result, more corporations are trying to influence the United States 
educational system and thus improve the skills of future workers (Coates, Jarratt, and 
Mahaffie, 1990). Also, policymakers and legislators, who shape the institutional environments 
and their policies, believe that students need certain skills in order to be effective citizens. 

The need for college graduates to communicate effectively is very important in our society 
where the daily operations and success of business organizations are contingent upon 
managing, making decisions, documenting, and reporting large amounts of complex 
information. In nearly eveiy study that has investigated the qualities employers most desire in 
their employees, "good communication skills, both verbal and wr'tten, rank high among the 
top priorities of those in business and industry" (Barabas. 1990, p. 9). For example, business 
executive^i, when asked about which college courses they considered as the best foundation for 
business leadership, rated oral and written business communication as very important more 
often than any of the other courses (Hildebrandt, Bond, Millen & Swinyard. 1982). Employers 
of civil and electrical engineering graduates rated speaking and writing as the most important 
areas of competence and these same areas were identified as the most deficient in these 
graduates (Kimel & Monsees, 1979). Human resource consultants in businesses indicated a 
clear consensus that companies with less hierarchies are requiring employers to be more 
autonomous (Cappelli, 1992). Leadership and communication skills are increasingly more 
important as managers are expected to supervise more people and in more informal reporting 
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arrangements. Communication skills in working teams become important as well as the ability 
to be flexible and open to changes necessitated by expanding fields of knowledge. Additional 
research has found that employees who write well have better prospects for advancement 
within business organizations (Anderson, 1985; Belohlov, Popp, & Porte, 1974; Cox, 1976; 
Stine & Skarzenski, 1979; Storms, 1983; and Van Dyck, 1980). While research has 
documented the importance of these skills for our future leaders in business and society, the 
specific nature and levels of attainment in these areas are unclean 

In order to define these skills, the appropriate stakeholders (faculty, employers, and 
policymakers) in higher education need to be consulted to determine rvhat specific skills are 
most important for college graduates to achieve. In short, to answer how we are doing as a 
nation in attaining Goal 5.5, we first need to define what particular skills and competencies rjre 
critical for students to obtain upon completion of their associate or baccalaureate degrees. 

In the fall of 1991 and 1992, NCES sponsored study design vyorkshops to identify the major 
issues and concerns related to an assessment of communication and critical thinking skills. For 
both workshops, national experts in assessment and college student learning including 
practitioners, reseai'chers, faculty, employers, and policymakers, were commissioned to write 
position papers. !n these documents, the authors identified the skills and levels of 
achievements that are important for college graduates, and they described a framework of 
methods for assessing these skills. These papers were distributed to reviewers and additional 
participants to obtain their feedback and evaluations in small working group sessions. 

These participants expressed many different views regarding the definitions and assessments 
of college student learning in the communication and critical thinking areas. There was no 
clear consensus about these skills. Workshop proceedings were produced by NCES (1994) as 
well as other documents that summarize the activities to date about a national assessment of 
college student learning. 

An inifial step in designing an appropriate national assessment of collegiate skills is a 
consensus-building process. Critical thinking, problem-solving, and communicafion skills are 
usually broadly understood. However, the specific skills that college graduates need in these 
areas are not clear. For example, Ennis (1987, p, 10) noted that critical thinking is "too vague 
to provide the schools and colleges with specific guidance/' The specific, important elements 
of critical thinking are often debated by philosophers, psychologists, and other educators. Very 
little research or formal studies have sought to include emiployers and policymakers in the 
same dialogue with faculty in explorations to define these important skills. 

In a series of four rccent public hearings, some individuals emphasized that in order for this 
effort to be effective, all constituencies must be involved in a consensus-building process to 
determine what skills college graduates need to set appropriate standards and definitions for 
achievement levels, and to review and evaluate approaches (Jones, 1993). Faculty and 
administrators representing the variety of institutions in different geographic locations as well 
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as employers, policymakers, institutional researchers, assessment experts, and higher 
education coordinating boards need to be included in this dialogue. 

The purpose of the study presented in this report was to determine if a consensus could be 
reached among faculty, employers, and policymakers about the important writing, speech and 
listening, and critical thinking skills that college graduates should achieve to become effective 
employees in the workplace and citizens in society. Reading and problem solving are two 
important abilities encompassed in Objective 5 of Goal 5. However, due to the limited scope of 
this current study, these two areas were not formally reviewed. 

We viewed writing, speech and listening, and critical thinking as generic skills that cut acror 
a wide range of disciplines in higher education as well as a wide range of work tasks within 
business organizations. By contrast, domain-specific skills and knowledge refer to those 
learned through academic courses that are often considered essential for work in a particular 
field. Domain-related content standards emphasize the knowledge and skills a student is 
expected to learn which characterizes a specific discipline. Domain knowledge of specific 
disciplines is important, but it is not the focus of this study. While domain knowledge is often 
regarded as the beginning point for successful performance in the work environment, it is the 
generic skills that count towards successful job performance over time (College Placement 
Council, 1994, p.27). Recent research on job performance consistently indicates the 
"inadequacy of attempts to itemize skills specific to an occupation; rather it has underscored 
the role of generic skills in rapidly changing job settings" (College Placement Council, 1994). 

In our study, over 600 participants volunteered their time and expertise to make judgements 
and evaluations about the relative importance of extensive lists of specific SKills. These 
individuals do not necessarily represent the universe of faculty, employers, or policymakers. 
However, a goal of this study was to expand upon the number of individuals involved in 
making these expert judgments. These individuals did possess the expertise to review these 
skills to determine their applicability and importance within the context of their own work or 
institufional environment. Through an iterative Delphi survey process (described more fully in 
Section III), these participants had the opportunity to agree or disagree with the importance of 
a variety of skills ranging from basics to advanced levels. They also received feedback about 
the overall average response from the initial survey and could revise their ratings as well as 
provide written statements about why they disagreed with certain average ratings of the enfire 
group of respondents. This empirical feedback gave the participants more information to 
evaluate their previous decisions. 

We identified areas of agreement as well as disagreement between the three groups of 
participants. Throughout this report, we have illustrated certain points with the actual 
participants' comments that may provide some insight concerning why they disagree about the 
importance of specific skills. We hope that these quoted comments expand upon the 
quantitative findings to suggest reasons for differing opinions or perceptions. Our goal is to 
provide an array of communication and crifical thinking skills that can be reviewed and 
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considered by faculty, employers, and policymakers. From these extensive lists of skills, 
individuals can adapt, modify, and decide which goals are most appropriate for their own 
college students. Through expanded and increased interactions among the relevant 
stakeholders of higher education, we can all work together to better prepare our college 
students to achieve the necessary skills for both the workplace and society. 



n. DEFINING THE IMPORTANT OUTCOMES 



There ai'e many frameworks ana research studies that articulate the potential array of writing, 
speech, and critical thinking skills that college graduates need to become effective 
communicators in society and stronger leaders or decision makers in business and industry. 
Many books and volumes of articles have been written by faculty members and other 
respected researchers. We reviewed the relevant literature in each skill area, including the 
commissioned papers and reviewers' comments from the recent NCES-sponsored workshops. 

Critical thinking is intricately linked with the ability to communicate effectively. However, in 
the areas of writing, speech, and listening, there are certain fundamental skills that college 
students need. These basic skills help college students to develop the technical abilities to 
receive and convey information. As college students develop their advanced abilities to analyze 
and evaluate, make judgements, and draw appropriate conclusions, then the overlap between 
critical thinking and the communication skills increase. Effective decision makers, managers, 
and communicators need to develop strong critical thinking skills. The mterconnections among 
these skills are further described in Section IV. 

The majority of the formal research studies concerning these skills are written by professors 
who study undergraduate students within the context of the college classroom. There are very 
few formal research studies that focus on the development of these skills within the context of 
the workplace. There are even fewer formal studies that investigate the communication and 
critical thinking skills that college graduates need to be effective citizens in society, especially 
upon the completion of their undergraduate degrees. 

In this section, we highlight the major frameworks and studies that help to define the essential 
skills that college students need. :vlany of the resources cited in this report provide a more 
thorough and greater detailed conceptualization and description of the various frameworks. 

A, Writing Abilities Framework 

There are three generations of research on composing (Faigley et aL, 1985). Each generation 
consists of certain trends in research methods and refinements in how composing is 
conceptualized. Writing experts propose various theoretical models, and many test their ideas 
by conducting fonnal research studies with college students. In our literature review, we 
examined sources that cut across these three generations. In this section, we provide examples 
and highlight the major theoretical models associated witn each generation. This h ief 
summary of the literature is intended to provide a context for the development of the survey 
instrument. Elements from these models provided key components for major categories used 
in the survey as well as the specific writing items included under each segment of the 
instrument. However, Faigley et al. (1985) provides a more thorough in-depth discussion about 
these various models. 
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The first generation of writing research studies posited that writing was a linear process with 
three main stages: pre-writing, writing, and re-writing. Many of the formal studies as outlined 
by Braddock, Lloyd-Jones, and Schoer (1963) aimed to test certain instructional strategies 
believed to improve the overall writing quality of students. Many of these studies compared 
two groups of students by using a pre-test and post-test research design. The goal was to 
determine if the group of experimental students taught by piuticular innovative or special 
techniques achieved better writing skills than the control group which did not experience these 
special methods. The academic performance of these two groups were compared in terms of 
the actual improvements from the time of the pre-test assignment to the post-test activity. The 
results usually indicated that the students in the experimental group wrote a higher quality 
post-test essay. 

The main model to emerge from this group of studies was delineated by the v/ork of Rohman 
and V/lecke (1964). They conceived of writing as an analogy of an "archetype of the plant'' 
(11). Writing was a process of development and a series of events that could be plotted on a 
line. It could be explored by examining the static yet structured, entire writing activity. The 
whole process could be analyzed point by point. They divided this line into two major 
segments: a portion of the process consisted of writing which occurs before words are written 
on paper, and the other part was what occurred after words are written. The former they 
labeled *'pre-writing,'' and the latter **writing" and "re-writing.'' Rohman and Wlecke 
conducted research in the pre-writing phase since they believed that this stage was the most 
crucial part of the writing process and had not received much attention in previous research 
studies. They envisioned the pre-writing stage as a discovery period where students 
^'assimilate" their subject to themselves. 

The college students in the "experimental group" of the Rohman and Wlecke study were asked 
to do three special things in their writing courses: maintain a journal describing their thoughts 
about what they did, practice some meditation principles, and employ analogies for the 
subjects of their essays. The control group did none of these tasks. The final results of their 
study indicated that many students in the experimental group enjoyed the course experience. 
They thought highly of the journal writing since it helf>ed them to better understand 
themselves and to work towards "self-realization." Students also liked the use of a concrete 
analogy as a method to organize their perspectives about certain subjects. Rohman and Wlecke 
concluded that students in the experimental group wrote better essays at the end of the 
instruction than the control group did. 

Other research studies built upon the work completed by Rohman and Wlecke. For example, 
Odell (1974) investigated the pre-writing stage and used certain procedures or sets of 
operations to aid college students with the pre-writing processes of exploring their own 
experiences and in the generation of ideas. Within this first generation of studies, some 
researchers viewed wrifing as an expressive process where ideaJly students discover meaning 
while writing. According to Faigley (1986), the criteria of good wrifing from the expressive 
viewpoint were integrity, spontaneity, and originality. 
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The second generation of studies focused more directly on the writer's strategies for 
composing and emphasized that planning occurs throughout the composing process. These 
studies demonstrated that the stages of writing are not clear cut and sequential (e.g., Emig, 
1964; 1971). There was a shift away from comparisons of teaching methods to the strategies 
that writers use in composing (Faigley et al.. 1985). Emig (1971) watched a small number of 
students as they wrote and asked them to speak about their ideas as they were writing. This 
research technique is known as **thinking-aloud protocols.'' Based upon the outcomes of these 
students' verbal expressions, Emig described the composing process as including 
considerations of wiidng contexts and identifying the nature of stimuli, prewriting, planning, 
starting, composing texts, reformulating, stopping, reflecting, and teacher influence (Emig, 1971). 

Much of the research in this generation also emphasized the revision process that students 
engage in to improve their writing. As Sommers (1980) noted, one of the main differences 
between speech and writing is that the possibility of revision exists for the written text. The 
linear models from the first generation tended to overlook the revision process. A linear model 
with discrete phases would view revision as an ^^afterthought.'' Revision could be simply the 
repetition of writing. Sommers sought to examine the writing process by comparing the 
revision strategies of experienced adult writers such as editors, journalists, and academics with 
the techniques of revision used by freshmen student writers. Each person in the study wrote 
three essays and rewrote each essay twice. Each writer was interviev/ed three times after the 
final revision of each essay. 

The final results indicated differences between the revision process of the student writers 
versus the experienced writers. The students viewed the revision process as a **rewording 
activity" because they believed that words were the important unit of writing. They 
concentrated on specific words divorced from their role in the text. Sommers noted the 
students are "governed, like the linear model itself . . . that prohibits logically needless 
repetition: redundancy and superfluity" (381). Students were most conce^'ned about repetition. 
Students did not see a need to revise if they believed they knew what they wanted to say from 
the very beginning. These students lacked **the procedures or heuristics to help them reorder 
lines of reasoning or ask questions about their purposes and readers" (383). 

The experienced writers in Sommers' study viewed their main goal in revising a written piece 
as discovering the **form or shape of their argument." Their first drafts were often attempts to 
define their ideas, and in the second draft, they would begin to develop more patterns of what 
to add or delete based upon their argument. The experienced writers also had more concern for 
their audience or readers. They would often use their reader's expectations to serve as a critic 
of their work. These expectations would shape how experienced writers would make 
subsequent revisions. Experienced writers viewed revision as a process of "discovering 
meaning altogether." 

A major model of ttie writing process to emerge from this second generation was developed 
from a series of research studies conducted by Flower and Hayes (1980a, 1980b, 1981). They 
studied the composing process and provide a theoretical foundation to explain the major 
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activities that occur when students write. While the first generation of studies was concerned with 
the linear stages of writing mostly influenced by the actual outcomes of the written product, the 
work of Flower and Hayes concentrates on the inner, tliinking processes of students as they write. 

The model by Flower and Hayes consists of several major units or parts of a system and how 
they are interrelated. The model begins with a **rhetorical problem" which involves the writer 
representing the problem that influences the v/riter's performance. From these representations, 
additional constraints enter upon the writing process. The expansion of the written text creates 
demands upon the writer. The writers use **knowledge stored in long-term memory" that 
includes their knowledge of the topic and the audience. Another factor is the writer's plans for 
dealing with the **rhetorical problem." The major writing processes in this model are 
**planning " ^^translating " and '^reviewing." Each major process included several sub-processes. 
For example, when students plan their writing they must also generate ideas and organize them 
into a structure. The identification of goals during the planning process is another key 
component. The next major process is arranging the ideas into ^Visible language." The act of 
reviewing information is another process that includes the sub-processes of evaluating and 
revising. Ultimately, the writer usually decides when to move from one process to another, and 
this '^monitoring" function is important and is determined by **the writer's goals and by 
individual writing habits or styles." Flower and Hayes view writers as constantly employing a 
group of cognitive processes as they 'integrate planning, remembering, writing and rereading'' 
(387). They do not find writing to be a sequential, step-like process. Writers are constantly 
planning and revising as they compose. However, Flower and Hayes assert that writing is an 
orderiy process in which writers develop goals and create '*a hierarchical network of goals that 
guides the writing process" (377). 

The third generation of research questioned some of the work by Flower and Hayes. This 
generation examines the composing process from the perspective of how a society uses writing 
rather than how the individual learner uses cognitive resources. In particular, Bizzell (1982) 
acknowledges that the determination of how students compose is important and provides a 
description of that process. However, she criticizes the Flower and Hayes research for 
neglecting to explore the social factors in the development of writing. The social factors 
involve the ''analysis of the conventions of particular discourse communities" (218), This 
analysis goes beyond examining the audience to exploring the expectations of that particular 
community to which the writer belongs and in which she or he shares some "virtues" by being part 
of a community. The discourse conventions are ''conditioned by the community's work" (219). 

In previous research, there v/as a lack of attention given to why writers are composing. The 
monitor in the Flower and Hayes model is responsible for making decisions about when to 
engage in certain composing processes, but it does not explain why the writer makes specific 
choices in different situations. Bizzell believes that composing models need to explain the 
social factors as well as the cognitive factors (as proposed by Flower and Hayes). The 
relationship between these sets of factors is an important consideration. Bizzell asserts that 
these factors need to be integrated to present a fuller picture of the composing process. She 
sunamarizes that the main factor missing from previous theoretical models is the "connection 
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to social context afforded by recognition of the dialectical relationship between thought and 
language.*' Bizzell proposes that the work by Vygotsky (1962) which explores the relationship 
of thought and language as the development of * Verbal thought" to be useful. Vygotsky's 
analysis implies that it is inappropriate to separate planning and translating as two separate 
processes. In order to fill in the empty translating part of the Hower and Hayes model, Bizzell 
suggests that socio-linguistics provides a way to analyze thinking and language use as 
conditioned by the social context. This line of thought led other researchers to pursue the 
sociology of knowledge (Bazerman, 1983) This perspective envisions writing as a social act 
that occurs m established contexts such as the academic disciplines. Bazerman (1981) 
examines several disciplines and demonstrates how differences in each discipline influences 
the expectations of the readers. His work indicates the differences in the nature of inquiry 
among these disciplines and in the outcomes of those inquiries. 

The research about writing over the last 25 years has focused primarily within the context of 
the particular writing students complete in school with an emphasis on essay writing in college 
composition courses. The three generations of research outlined above all contribute to an 
understanding of what characteristics distinguish good essays from poor ones. They also 
provide knowledge about the composing strategies of the most proficient student writers as 
contrasted with the less proficient ones. In many cases, college students were asked to respond 
to exam questions in which the writers assume they were supposed to demonstrate to their 
readers (the college faculty) that they knew what the readers expected ihem to learn. However, 
there are real questions that employers may have in which they are asking their employees to 
give information to their readers and to make reasoned judgements. 

In general, college professors are interested in kow students get the results and with the 
process of writing. A few research studies (e.g., Barton & Barton, 1981; Paradis, Dobrin, & 
Miller, 1985) conducted in the work environment suggest that supervisors in industry were 
mainly concerned with the final results and specific recommendations. While some faculty 
may emphasize the problem-solving process involved in writing an essay, employers "want 
information that will help them m<ike decisions, which in turn will achieve organizafional 
goals" (Barabas, 1990, xxii). 

B, Speech Communication Framework 

The NCES-commissioned papers in the speech communicafion area, reviewers' comments 
about these papers, and the NCES conference proceedings v/ere reviewed in preparation for 
the development of the speech communication goals inventory. In addition, we conducted a 
literature review that examined the important skills idenfified according to faculty experts, 
policymakers, and employers. Many of these skills drawn from previous research studies were 
incorporated into the inventory that we developed. In particular, the organizational framework 
utilized in this study was drawn from the framework adopted and endorsed by the Speech 
Communication Association (SCA). In this section, we provide a brief overview of the major 
frameworks and research that has been conducted in the area of speech communication. 
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Communication experts have generally found it difficult to isolate specific listening and 
speaking skills which should be possessed by competent communicators (Rubin, 1982). 
Furthermore, there is little consensus on "the constituent definition of oral communication 
competence nor the operational skills which the construct represent'* (McCroskey, 1982, p. 9). 
Although a substantial amount of material has been written to offer guidelines for the 
assessment of students' communicative competence, "almost as many definitions of the 
concept of competence exist as there are authors. These definitions are not specific about the 
components of competence and leave unoefined certain crucial concepts which are necessary 
to understand the nature of communication competence" (Bostrom, 1984, p. 11). While many 
faculty, employers, and policymakers would agree that college graduates should learn 
behavioral and cognitive abilities to become competent communicators, it is difficult to reach 
an agreement about the specific nature of these skills. "On most campuses we cannot even get 
our faculty to agree on what should be included in our basic course!" noted McCroskey (in 
Bostrom, ed., 1984, p. 267). 

The acquisition of basic communication skills is necessary for effective participation in society 
and in the workplace. A number of scholars (e.g.. Dance & Larson, 1972; Duncan, 1968; 
Reusch, 1957) have suggested that there is a relafionship between individuals' abilities to use 
the communication process and their abilities to function in society. In order for individuals to 
function effectively in society, they need to achieve a certain level of competence in the use of 
language and nonverbal behavior for the purpose of communication. This competence has 
been called "communicative competence." 

The challenge in defining "communicative competency" seems to stem from the differences 5n 
three major perspectives: cognitive, behavioral, and social cognitive. The choice of perspective 
influences the definition and is important in the applicafions of the concept. The cognitive 
perspective usually consists of definitions that view competence as a mental phenomenon that 
is disfinct and separate from behavior. Chomsky (1965) developed a perspecfive of linguistic 
competence that focused on the nature of linguistic knowledge which is the appropriate 
domain of inquiry rather than performance. Behavioral considerations were placed outside the 
domain of linguistic theory and research. Chomsky described and defined linguistic 
competence as the individual's knowledge of the structure of language. Scholars who advocate 
this concept believe that "the goal of competence theories is not the explanations of events or 
processes, but rather the discovery of the cognitive structure and mental representations that 
underlie events" (Wiemann & Backlund, 1980, p. 187). The major goal here is to create a set 
of idealized, formal rules that would underlie behavior. 

The second perspective is a category of definitions that emphasize the performance aspect of 
communicative competency. This view refers to actual communicative behavior Competence 
is viewed as a function of social skills and social outcomes rather than one's own perception of 
competence. This perspective suggests that communication competence is dependent upon 
certain skills that enable individuals to achieve their goals. In this area, some scholars have 
defined competence in the form of lists containing minimal levels of achievement (Allen & 
Brov*/n, 1976; Bassett, Whittington, and Stanton-Spicer, 1978; Wood, 1981; Wood et al., 
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1977). These lists have been base ' upon extensive reviews of literature. For example, Allen 
and Brown (1976) identified five performance dimensions of communication competence: 
controlling, feeling, informing, ritualizing, and imagining. In a similar manner, Ruben (1976) 
believed that seven dimensions were critical: display of respect, interaction posture, orientation 
to knowledge, empathy, interaction management, tolerance for ambiguity, and self role- 
oriented behavior. Wiemann (i977b) proposed a model with five major components including 
interaction management, empathy, behavioral flexibility, social relaxation, and affiliation/ 
support. Bassett, Whittington, and Stanton-Spicer (1978) identified four specific competence 
areas which comprise the skills that high school graduates should possess: communication 
codes, oral message evaluation, basic speech communication skills, and human relations. 
These competencies are applied to three different contexts including occupational, citizenship, 
and maintenance. Daly (1992, p. 26) noted eight major communication categories: 'influence 
and compliance gaining as well as resisting influence attempts, information seeking and 
giving, affinity seeking and maintenance, conflict management, expressing oneself, 
explanations and accounts, coping strategies for problematic communication events, and 
negative behaviors that nonetheless, mark effectiveness.*' Despite these numerous 
communication competence studies, several categories appear repeatedly in the researchers' 
lists. These include empathy, interaction management, and behavioral flexibility. Empathy is 
usually considered to be the ''other-oriented'' component of competence. These abilities enable 
an individual to evaluate another's intentions and imagine oneself into the thinking, feeling, 
and acting of another individual. This skill provides knowledge for making decisions among 
communicative choices. Interaction management includes such skills as appropriate control 
over topics initiated and discussed, listening, taking turns appropriately when speaking, and 
other skills related to interactional goals. Behavioral flexibility refers to the individual's 
adaptation to the situation by making appropriate communication choices. 

The framework proposed by Bassett et al. (1978) was adopted and endorsed by the Speech 
Communication Association (SCA). Rubin (1982) built upon the Bassett et al. framework and 
developed a communication competency assessment instrument that seeks to determine 
communication skills in the educational setting. All of the competencies in this instrument 
relate to the student's ability to function "in specific educational environments: in classrooms 
and with instructors, fellow students, and academic advisers" (Rubin, 1982, p. 21). 

The third, more recent perspective, is social cognition. This view is "a process of representing 
knowledge about people and their relationships. This knowledge is generally believed to be 
acquired through participation in ongoing, dynamic interactions" (Sypher, 1984, p. 1 13). 
Roloff and Berger (1982) view social cognition as thought directed toward interaction. 
O'Keefe and Delia (1979) have asserted that interpersonal variables form the basis for 
communicative choices. This reasoning suggests the need to evaluate communication 
competence across situations since individuals may have cognitive and behavioral skills that 
lead to effectiveness in one setting but not in another (Sypher, 1984). 

While the multitude of literature provided many skill statements, there were differences in the 
specific views of communication competence as noted above. The framework adopted for the 
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study described in this report was taken from the work originally conducted by Bassett et al., 
and subsequently adopted by the SCA as well as the foundation for an assessment instrument 
developed by Rubin (1982). The four major categories are basic speech communication skills, 
communication codes, oral message evaluation, and human relations. Specific examples within 
each major category were derived from a variety of relevant studies pertaining to college 
students' communication skills. 



C. Critical Thinking Abilities Framework 

The NCES-commissioned papers in the critical thinking area, reviewers' comments about 
these papers, and the NCES conference proceedings were reviewed in preparation for the 
development of the critical thinking survey. In addition, we conducted a literature review that 
integrated the important skills identified by faculty experts, employers, and policymakers. 
Many of these skills were drawn from previous research studies. In particular, the framework 
for this survey builds upon the work completed by Facione (1990). While the organizational 
structure draws from Facione's major categories and sub-units derived from his previous 
Delphi study, the individual items in each section reflect additional skills considered to be 
important by a diverse group of individuals. 

There are many definitions and conceptualizations of critical thinking. Some definitions are 
implicit within national standardized instruments such as the critical thinking module of the 
Collegiate Assessment of Academic Proficiency (CAAP) program developed by the American 
College Testing (ACT) Company. Other expert scholars propose theoretical models and 
frameworks that more fully outline their conceptualizations. In this section of the report, a brief 
overview of some definitions of critical thinking that arc embedded in models will be presented. 

Robert Ennis (1987) defines crifical thinking as "reasonable reflective thinking that is focused 
on deciding what to believe or do" (p. 10). He has developed an itemized list of skills and 
dispositions that are major components of crifical thinking. Ennis believes that critical thinking 
is closely connected with creative thinking and problem solving. He views critical thinking as 
a rational process. 

Critical thinking according to Richard Paul (1993) is "disciplined, self-directed thinking which 
exemplifies the perfections of thinking appropriate to a particular mode or domain of thinking" 
(p. 462). It is an art of thinking about one's thinking in order to make it more clear, accurate, 
or "more defensible." This type of thinking requires "the ma.stery of intellectual skills and 
abilities." Paul considers the extent to which critical thinking is dependent on the ability of an 
individual to develop insight into egocentric and ethnocentric thinking, the tendency towards 
self-deception, and the growth of a moral character. His definitions include the ability to 
criticize oneself However, the specific means to achieve this ideal state is unclear. 

McPeck (1990) views crifical thinking as important within the context of the disciplines and 
the knowledge within those fields. Only through an immersion in the disciplines can 
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individuals fully develop their abilities to think critically. McPeck includes actions as well as 
beliefs in his definition. He also advocates the importance of "reflective skepticism** which 
encourages the individual to find the faults of others. 

Siegel (1988) notes '*a critical thinker is one who is appropriately moved by reasons: she has a 
propensity and disposition to believe and act in accordance with reasons; she has the ability to 
assess the force of reasons in the many contexts in which reasons play a role'' (p. 23), SiegeFs 
theory asserts a close connection between critical thinking, rationality, and problem solving. 

This brief overview of various definitions of critical thinking indicates that philosophers and 
cognitive/educational psychologists have some distinctions they maintain within their 
conceptualizations of these thinking abilities. These distinctions reflect differences in the 
approaches to methodologies, the study of modes of thinking, language use, roles of values in 
thinking, and the view of the teaching process. For example, Paul (1993) notes that cognitive 
psychologists approach thinking descriptively while philosophers approach it normatively. 
Psychologists tend to focus on "expert versus novice thinking, intradisciplinary thinking, and 
monological thinking" while philosophers concentrate on "rational, reflective thinking, on 
interdisciplinary thinking, and on multilogical thinking" (p, 448). 

Overall, psychologists examine the cognitive structure and activities of the mind (Young, 
1980), From this perspective, critical thinking "can be characterized by the ways in which the 
contents and mechanisms of human cognition are involved in the solution of problems and the 
making of decisions and judgments. Teaching is concerned with the development of these 
contents and mechanisms*' (p. ix). While psychologists emphasize the importance of 
cognition, contemporary philosophers are concerned with the identification of methods (such 
as comparison or classification) that allows individuals to solve the abstract and practical 
problems of life. 

There is some disagreement as to whether critical thinking is subject-specific or generalizable 
(Ennis, 1987; Johnson, 1992; McPeck, 1990), Can critical thinking skills be transferred from 
one subject to another? McPeck (1990) emphasizes that generalizable critical thinking skills 
do not exis^ and that thinking is always about a subject. Therefore, thinking detached from a 
discipline or subject cannot exist. The unresolved issue is how much knowledge of content is a 
significant factor in critical thinking. There is a general agreement that a student's familiarity 
with the subject matter plays an important role in the student's performance on thinking tasks 
in that area (Ennis, 1992; Ennis et al., 1987), Furthermore, Perkins (1985) stresses that many 
intellectual skills are context-specific. In a similar manner, Paul and Nosich (1991 ) emphasize that 
a critical thinker should consider the epistemological structures and intellectual standards within the 
confines of the discipline in which a problem is being addressed. This argument assumes that there 
can be no general critical thinking skills, Paul ( 1985) notes this is similar to assuming that when we 
write or speak, there can be no teaching of general wrifing or speaking skills. 

In response to the claims that crifical thinking is subject specific, Ennis (1987) suggests that . 
there are general principles of critical thinking that bridge subjects and have application to 
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many subjects. Critical thinking may transfer to new situations. Ruggiero (1988) indicates that 
comparison of approaches of thinking instruction designed for different subjects reveal some 
variations to thinking instruction and some variations in terminology, but teach essentially the 
same cognitive skills. Ruggiero (1988) indicates that a number of approaches to develop 
thinking skills in a particular discipline have been successfully applied to other disciplines as 
well. According to Facione (1990) critical thinking has applications in all areas of life and 
learning. He cautions that ''although the identification and analysis of critical thinking skills 
transcend, in significant ways, specific subjects or disciplines, learning and applying these 
skills in many contexts requires domain-specific knowledge. This domain knowledge includes 
understanding the methodological principles and competence to engage in norm-regulated 
practices that are at the core of reasonable judgments in those specific-contexts" (p. 5). 
However, the panel of experts who participated in Facione's Delphi study recommended that 
becoming skilled in critical thinking requires learning to use these abilities effectively in many 
different contexts. Students have multiple roles in society, particularly as citizens or employees 
who require the development of critical thinking skills and dispositions. 

Despite the differences in these conceptualizations, critical thinking is generally thought to 
consist of two main general components, a disposition to think critically and a cognitive 
component. There was some consensus from Facione's Delphi study that critical thinking 
requires the use of cognitive abilities including the application of techniques, rules of 
reasoning, skills, or procedures. Facione's study was conducted with a group of 46 national 
experts, mostly faculty members in education, social sciences, or physical sciences. Through 
several rounds of surveys with this interactive panel, a consensus was reached concerning the 
ideal skills associated with a good critical thinker. An outcome from Facione^s study was a 
classification of important skills and sub-skil's which included interpretation, analysis, 
evaluation, inference, explanation, and self-regulation. This classification system was the basis 
of the Delphi study that we conducted with faculty, employers, and policymakers. 

The experts in Facione's study did not view critical thinking as a body of knowledge delivered 
to students in a single additional course. *'Like reading and writing, critical thinking has 
applications in all areas of life and learning. . . The instruction of critical thinking can occur in 
programs rich with discipline-specific content or in programs which rely on the events in 
everyday life as the basis for developing one's critical thmking'' (p. 5). 

Although a student m.ay possess the cognitive abilities to think critically, the individual may 
not be motivated or inclined to use them. Therefore, scholars and researchers assert that 
critical thinking also includes an affective dimension often referred to as dispositions or traits 
of mind, which characterize a critical thinker's way of behaving (Ennis, 1987; Facione 1990, 
1992; Halpem, 1992; Paul, 1992; Paul & Nosich, 1991; Perkins et al., 1993). Dispositions are 
critical to advanced reasoning. For example, a study found that while people may have the 
ability to generate arguments on the side of an issue opposite their own, they may not be 
inclined to explore the other side unless prompted or motivated (Perkins, Farady, & Bushy, 
1991). Students can be intrinsically motivated when they perceive a situation as an issue. A 
key factor is curiosity which is triggered when students experience gaps in their own 
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knowledge. However, there are some practitioners and scholars who do not believe 
dispositions are part of critical thinking. In Facione's study, roughly one-third of the panel of 
experts viewed critical thinking as referring only to the cognitive skills and dispositions, but 
not to affective dispositions. This minority assert that good critical thinkers are those 
individuals who have certain skills and habits. If people are good critical thinkers, then they 
use their skills appropriately since good thinkers have some or most of these affective 
dispositions. 

There is agreement among scholars that critical thinking should include meta-cognitive or self- 
monitoring skills (Beyer, 1988: Facione, 1990: Halpern, 1992: Marzano et al., 1988; Perkins et 
al., 199 1 ; Swartz & Perkins, 1990). "Meta-cognition refers to what we know about what we 
know, or, in more formal language, our knowledge about knowledge" (Halpern, 1984, p. 15). 
Meta-eognition is being aware of one's thinking as one performs specific tasks and then using 
this awareness to control what one is doing. 

Scholars of critical thinking emphasize that a student's thinking should also meet intellectual 
standards, also called norms or criteria of good thinking (Lipman, 1988, 1991; Paul, 1992, 
1993). These intellectual standards implicit in critical thinking include clarity, relevance, 
accuracy, fairness, completeness, precision, depth, breadth, and adequacy. Intellectual 
standards represent legitimate concerns irrespective of the subject being explored or the 
question at issue (Paul, 1992). 

Critical thinking is related to problem solving. However, in this report we make a distinction 
between these two skill areas and focus specifically upon critical thinking. Critical thinking 
usually involves reasoning about open-ended or 'MU-structured" problems while problem 
solving is usually considered to be narrower in scope (Kurfiss, 1988). In problem solving as 
studied in cognitive settings, there is often only one correct answer to a complex problem and 
only a limited number of options or sometimes only one method to solve the problem. These 
problems are usually referred to as "well structured." The goal is to discover and implement a 
solution. Problem-solvers generally develop and refine their problematic situation. Then they 
analyze their current state, identify constraints, gather information, and generate and test at 
least one hypothesis until their goal is attained. Critical thinking involves reasoning about 
issues that have no single solution. Here the goal is to construct a realistic representation of the 
situarion or issue tnat could be presented in a convincing argument (Kurfiss, 1988). While the 
representation of the model can be outlined in the form of a proposition that claims to account 
for all available information, it cannot be proven or tested. Therefore, it must be supported 
with appropriate reasoning and evidence. The process of developing support for a position 
most clearly distinguishes critical thinking from problem solving (Kurfiss, 1988). Critical 
thinking tends to be associated with the social and behavioral sciences while problem solving 
is often associated with mathematics and sciences. 

Critical thinking is also related with other communication skills. Halpern (1984), Chaffee 
(1990), and Swartz and Perkins (1990) in their critical thinking textbooks underscore the 
relationship between literacy abilities and thinking. Relevant critical thinking skills in this area 
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include those that are necessary to comprehend r^d defend against the persuasive techniques 
that are embedded in everyday language. These skills also include listening and speaking skills 
used to clarify thinking or increase specificity. Furthermore, critical thinking includes critical 
reading and writing skills (Browne & Keeley, 1981; Paul & Nosich, 1991). For example, 
reading and listening skills relate to the individual's ability to assess the purpose, biases, and 
credibility of a speaker or author, and the ability to accurately identify the problem or issue 
being evaluated and the underlying assumptions of one's views (Halpem, 1992; Paul & 
Nosich, 1991). All of these communication skills are interrelated. Therefore, we included 
critical thinking aspects in the writing goals survey and the speech communication survey. 
Since there are so many ideal skills associated with critical thinking, we decided to create 
a separate survey to explore further the specific nature of what constitutes making reasoned 
judgements. 
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A* Delphi Technique 

We employed a Delphi survey technique to gather feedback from faciihy, employers, and 
poHcymakers. This approach has been used for planning in many higher education settings to 
improve communication and reach some consensus about a variety of issues (Uhl, 1983). Uhl 
(1971) conducted a Delphi survey to identify institutional goals and obtain a convergence of 
opinion from diverse constituent groups. In a similar manner. Fox and Brookshire (1971a, 
1971b) used the Delphi technique to obtam agreement among faculty on what should be 
considered for effective teaching. 

For this current project, the Delphi technique helped us to develop and determine the range of 
possible skills and competencies for critical thinking and communication areas. This tool was 
particularly useful since the identification of these skills involves the collective judgment of 
diverse groups of stakeholders. The skills and competencies are more likely to be accepted if 
more people participate in its exploration than would be possible in personal or group 
interviews and meetings. Through the Delphi process, paiticipants have adequate time for 
thinking and reflecting about these potential skills. 

A diverse group of stakeholders participated in the Delphi process which provided some 
insights into the underlying assumptions leading to different judgments. Through this process, 
there was a consensus about the importance of certain skills from the initial round of surveys. 
This consensus was determined by the statistical procedures described in the data analysis 
section. Throughout the report, we highlight the areas of consensus. However, this does not 
mean that all participants in this study were in total agreement about the importance of certain 
skills; rather, the statistical procedures we used indicated that there were not significant 
differences in the responses between the groups nor within particular groups. 

In the second round of surveys, participants reached some consensus about certain skills where 
no agreement existed from the initial round of surveys. This means that after the participants 
re-evaluated their positions, they reached an agreement about additional items which was 
again determined by the statistical procedures we ernployed. In the second round, participants 
were given the average group response for each item and were asked to re-evaluate their 
position. By providing the group average and asking for more reflection about certain items, 
the Delphi technique encourages consensus from groups of participants. However, if 
individuals disagreed with the group average, they described their reasons for having a 
different position in this second survey. 

In this report, we will highlight minority and differing perspectives as well as areas where 
there are agreements. In summary, the Delphi method provided information about: (1) the 
range of ideas about the necessary competencies and skills, (2) a priority ranking of these 
competencies, and (3) some degree of consensus about these skills. While the Delphi technique 
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has certain advantages for this project, the main limitation is that personal interaction among 
participants was minimal. Therefore, the advisory boards and focus groups played a key role in 
helping us to refine our survey instruments before they were sent to the larger group of participants. 



B. Development of Goals Inventories 

We used three major strategies to develop the goals inventories. First, we reviewed the relevant 
literature in each skill area (writing, speech and listening, and critical thinking) including the 
commissioned papers (sponsored by NCES), reviewers comments about these papers, and the 
NCES workshop proceedings. Then we synthesized the skills that were repeatedly identified 
according to faculty experts and employers. From these extensive lists, we identified the 
specific skills that were mentioned by more than one author. We grouped sets of skills together 
under certain major headings that were derived from the literature. Then we used these sets of 
skills to develop a critical thinking goals inventory and separate surveys for speech and writing 
skills. Each inventory consisted of an organizational framework drawn from the literature. 
Therefore, this current study sought to build upon the findings of previous scholariy work. 

This current study required the participation of those individuals closest to the educational 
process and outcomes in order for the results to be useful. Therefore, our second strategy was 
to form an advisory board for each skill area. We invited national faculty experts, employers, 
and policymakers to join each advisory board. Some of these individuals were selected since 
they had participated in prior NCES workshops. We believed this continuity was important so 
that we would build upon the previous exploratory workshops already completed. However, 
we also invited new individuals who had never participated in these discussions to join each 
board. These new individuals (from both community colleges and universities) were selected 
because they had published and conducted a great deal of research in the skills areas. We 
wanted to make linkages and connections with other research centers that focus on public 
secondary education (in these areas) as well as with professional associations. Therefore, 
advisory board members were selected from these organizations. The main role of this 
advisory boar d was to critique the draft versions of the goals inventories. 

Our third strategy was to convene a focus group consisting of faculty, employers, and 
policymakers in each skill area. We invited some individuals from the NCES workshops. 
However, we also identified additional participants who had experience with instruction in 
communication and critical thinking. The purpose of the focus group was to review the 
instruments and identify ambiguities or areas that were unclear. We wanted to use clear, 
precise language that all participants would understand. 



C. Feedback from Advisory Boards and Focus Groups 

Each advisory board was convened at least once through a phone conference call. They were 
provided with draft versions of the survey instruments. During the conference call, they were 
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asked to critique tliem and offer recommendations about how to strengthen each instiniment. 
Each board member offered constructive criticism and feedback. 

We refined the instruments after we received feedback from the advisory boards. Then we 
convened the focus group members. Each group meeting lasted approximately three to four 
hours. We asked them to review a draft version of the instrument along with its accompanying 
cover letter. We incorporated the feedback received from the focus groups. Then we sent 
another version of the instrument to both the advisory board and the focus group members. 
Originally, we had planned to continue to get feedback throughout the development phase of 
the surveys from only the advisory board. However, many focus group members wanted to 
remain involved with the project, and, therefore, they reviewed the instrument an additional 
time. We consulted with individual advisory board members further as we refined the 
instrument by making individual phone calls to clarify points and gather more feedback. 

Both advisory boards and focus groups made suggestions to strengthen the instrument. The 
advisory boards recommended organizational frameworks. Both groups offered ideas about 
items that should be added to the survey and noted areas where there was a duplication of 
items. They also recommended clearer wording to reduce some ambiguous terms. The focus 
groups, in particular, noted in some cases "academic jargon" which contained terms they did 
not understand. These participants also suggested a clearer set of instructions for the three 
major groups of respondents. In some cases, they recommended a slightly different 
organizational structure for the major categories in the surveys. For example, in the writing 
survey, the majority of individuals believed that a separate section about the generation of 
ideas was not necessary and could be incorporated into the pre-writing section of the survey. 
However, many individuals thought a separate section on collaboration was important. 

After the focus group members completed the surveys, we discovered that most individuals 
ranked all items as extremely important within the context of the five choices they were given 
on the rating scale. Many members suggested changing the scale to a choice of nine ratings 
ranging from "extreme importance'' to "medium importance" to "no importance." When we 
sent the surveys to focus group members with the new nine rating scale, there was a broader 
range or dispersion of responses. 

There were a few areas of common concerns and areas of debate among the three different 
advisory boards and the three focus groups. Since some groups raised similar issues, we will 
highlight these concerns. 



C.i. Defining the Context 

Some individuals were troubled by the very notion of trying to reach a consensus across a 
diverse group of faculty, employers, and policymakers at a diveise group of organizations. For 
example, one advisory board member stated: 
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Different audiences have different cultural and communication norms, which implies 
differences in background and in the topic. But different organizations within a society 
have different cultural and communication norms; a successful memo at Citibank will not 
work at IBM. In fact, there may be more important differences between memos in these 
two settings than there are between memos written for French versus Japanese audiences, 
which themselves are far from homogeneous. 

They emphasized that college graduates use different communication skills in a diverse 
number of contexts. 

Some individuals noted that the responses for certain items on the survey instruments 
depended on the specific situation and the specific type of communication that the college 
graduate was attempting to complete. Others were troubled by the definition of a college 
graduate to include both four- and two-year degree recipients in the same survey instrument. A 
focus group participant noted that "there is a development difference between students who 
finish with a two-year degree and a four year degree." However, other individuals cautioned 
about the separation of the associate degree recipient versus the baccalaureate degree graduate. 
A focus group member stressed: 

I think the goal [of this project] is to improve teaching and learning. It doesn't matter if 
you're a two-year, four-year, five-year student. You're in the context of learning to 
become someone who has and can demonstrate these skills and abilities. I'd like to 
disengage the discussion from being so dependent on two-year and four-year programs. 

Another focus group participant agreed and stated: 

It's a little dangerous to totally distinguish between four-year and two-year graduates; 
certainly some of the two-year graduates we have are adult learners who have different 
expectations than those who are nineteen years old, but I think the same could be true for 
a four-year program. 

This debate focused on whether college graduates should be separated between associate and 
baccalaureate degree students or integrated together. 

Some individuals believed that important goals could be identified across contexts. As one 
professor stated: 

Let's look at these [writing goals] as a group, and as a curriculum committee let's go 
through them and figure out is this really what we want, is this what should be happening, 
is this what we can do. I would hate to see us deconstruct it [the writing goals] so much 
that nothing gets sent out. I think it could be a really useful learning tool and a very 
empowering tool to start a conversation in 600 institutions, if they're not already started 
in these areas. 
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Some individuals were most concerned about capturing every potential context and every type 
of differentiation while others were concerned about succumbing too much to reductionism so 
that the qualities sought from college graduates get lost or are never explored because it is too 
complex a process. 



C.ii. Linking Results with Instruction in the Classroom 

Both advisory board and focus group members emphasized the importance of linking the 
results from this current project to making improvements in classroom instructional strategies 
and assessments of student learning. A focus group member stressed that the most important 
question is "how do you raise the consciousness of people who are in the classroom both 
teaching and learning to affect the development of these skills which is really what it should be 
like." Another participant noted that the implementation of these goals is important, but it is a 
major challenge to structure classes successfully to achieve these goals. Many individuals 
believed it is crucial to build upon the results from this study to make improvements in the 
classroom. Without a link to the teacher in the classroom, there was a perception that some 
faculty would teach only some of these skills while perhaps missing important areas. Some 
participants noted that learning what each particular group of respondents believes are 
important skills could be useful "in terms of confirming why there is a growing sense of 
frustration and wanting to somehow take a look at those gaps." An individual remarked: 

I think that one of the really exciting and interesting things about this project is that it is 
going to help all of us better understand what kinds of things we want our students to 
learn. Not just to satisfy me, but to satisfy you and everyone else. It is so we don't have 
the discrepancy between the faculty saying, *yes, you are educated* and then they go out 
in the world and business says *you cannot do anything/ 

Another professor concurred and remarked: 

Tve been involved as an academician with practitioners out there. Tm always brought up 
short of reality. I think there's some real value here in communicating with each other and 
getting perspectives on this because I think you really do get a very limited, and 
sometimes myopic, perspective if you just talk to the academics. 

A faculty member emphasized that the writing goals inventory represented "my wish list of 
what I hoped happened when teaching writing. I spend a lot of time on the road talking to 
faculty and there's a lot of faculty teaching writing who aren't doing half of these things." 
Others thought the goals in the surveys encompii ,ed those things that they hope to teach and 
that they hope college students will leam as a result of their teaching. 

A debate ensued within each group about whether the main question was what skills college 
graduates should have versus what skills they actually do possess. The majority of reviewers 
felt they lacked sufficient knowledge to determine or identify whether their own college 

21 



graduates actually attained certain skills. In addition, they believed there was a great deal of 
variation in the levels of students' competencies. They thought that definitions about what 
skills college graduates should achieve would be within their scope of expertise to evaluate. 
Therefore, both the cover letter and the instructions on the survey sent to participants 
emphasized that we were seeking feedback about what skills college graduates should possess. 



C.iii. Expanding the Dialogue 

A common theme cited by participants was the importance of expanding the dialogue. Some 
participants recommended that recent college graduates should be asked about the levels of 
skills they have achieved and whether they are adequately prepared for their current 
professional positions. Other participants emphasized the need to include faculty from all 
disciplines. For example, the development of communication skills are not the sole 
responsibility of writing professors or speech faculty. 

An underlying current in the perceptions of some individuals was a deeper concern about how 
the results from this project would be used in the future. We emphasized that this current 
project was a small study designed to determine the extent to which there is or is not a 
consensus about the importance of various skills and abilities that college graduates should 
possess. We informed all focus groups an i advisory boards that this is not a large nor long- 
term, conclusive study. Instead, we stressed that this was an initial step to gather feedback 
from a diverse audience about the importance of skills associated with effective writing, 
speech and listening, and critical thinking. The purpose of this study is to question to what 
extent do the expectations of experts in the academic fields actually bear some resemblance to 
the expectations of employers in the workplace and policymakers in state level higher 
education coordinating boards or accrediting associations. 



D, Selection of Study Participants 

A major goal for the selection of study participants was to identify and choose colleges ?.nd 
universities that reiflected a national distribution of students. We used the enrollment data from 
the 1990-1991 enrollment survey for the Integrated Postsecondary Education Data Systems 
(IPEDS). The sample universe included all institutions that were part of the IPEDS with two 
exceptions. No proprietary schools were included since they do not award associate or 
baccalaureate degrees. We also excluded schools that enrolled less than 500 students since it 
would be difficult to find the appropriate numbers of faculty, employers, and policymakers to 
participate in this study. 

We used a three step stratified random selecrion technique. Groups of institutions were first 
strarified by using the Carnegie Classifications (1987) to categorize institutions. The categories 
of research I and 11, doctoral grandng I and 11, comprehensive I and II, and liberal arts I and II 
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were each collapsed into the categories of research, doctoral granting, comprehensive, and 
liberal arts. The Carnegie Classifications for other professional institutions (such as teachers 
colleges or religious schools) were collapsed into one category. This step yielded six cells. 

We conducted the second stratification by using the regional accreditation codes which 
includes six different organizations (e.g.. New England Association of Schools and Colleges. 
Middle States Association of Colleges and Universities). This stratification was employed to 
obtain institutions that reflected different geographic regions in the United States. This step 
yielded a six (Carnegie Classification) by six (accrediting agency) matrix or 36 cells. We 
conducted the third stratification by employing the type of control, either public or non-profit 
private. This step yielded a final 6x6x2 matrix of 72 cells. 

We determined institutional size by a median split within Carnegie Classification categories 
with those institutions at or below the median labeled as small and those with enrollments 
greater than the median labeled as large. This was used to check for variarion in student 
enrollments across the selected institutions. 

To determine the proportion of the database institutions in each of the 72 cells, we used a 
crosstab procedure. The resulting percentage was used to determine the number of institutions 
to draw from each cell by multiplying 150 (the initial number of institutions needed) by the 
percentage in each cell. AH cells were represented by at least one institution. 

Random numbers from the uniform distribution were assigned to each institution within each 
cell. The institutions were then sorted by the random numbers, and insitutions drawn with six 
percent coming from each cell. The final yield was a total of 150 institutions. 

Even though academic administrators at 150 different institutions were invited to participate in 
this study and nominate colleagues, ultimately administrators from 78 institufions responded 
positively with completed nomination forms (see Table 1). The institutions in the sample 
differed slightly from the population of potential schools. While the population of community 
colleges is approximately 41.3 percent of the total insfitufions in the United States, only 28.2 
percent actually agreed to participate in our study (see Table 2). Slightly higher proportions of 
research, comprehensive, liberal arts, and doctoral granting institutions responded to our 
invitations. None of the invited professional schools (such as teachers colleges or business 
schools) agreed to participate although only 4.3 percent of the population of institutions are in 
this category. There was a fairly even distribution of the selected institutions across the six 
regional accrediting areas (see Table 3). A slightly higher proportion of institutions accredited 
by iMiddle States and Northwest Association participated while a slightly lower proportion 
from North Central and the Southern regions were actually involved. 

The group of institutions selected closely approximated the population of institutions in terms 
of public versus private control (see Table 4). However, a higher proportion of institutions with 
large student enrollments participated than small institutions (see Table 5). 
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Table 1. Participating Institutior,s by Carnegie Type 



RESEARCH 


Cornell University 
Howard University 
Northwestern University 
Texas A&M University 


University of California, Los Angeles 
University of Hawaii, Manoa 
University of New Mexico 


DOCTORATE 


The American University 
Biola University 
Drake University 


Georgia State University 
The University of Akron 
University of Notre Danie 


COMPREHENSIVE 


Aquinas College 

Austin Peay State University 

Black Hills State University 

Bloomfield College 

Brigham Young University, Hawaii 

California State University, Fullerton 

Eastern New Mexico University 

Freed-Hardeman University 

Gonzaga University 

Ithaca College 

Millikin University 


Rochester Institute of Technology 
University of Alaska, Fairbanks 
University of Baltimore 
University of Massachusetts, Boston 
University of Puget Sound 
University of Southern Colorado 
University of Texas, El Paso 
University of Texas, Permian Basin 
University of Wisconsin, La Crosse 
Xavier University 


continued on next page 



The academic vice-president or tlie equivalent senior academic administrator (such as provost 
or dean of faculty) at each institution was contacted with a letter of invitation to participate in 
the study. The cover letter explained the purpose of the study We asked each administrator to 
nominate three faculty members each in the areas of writing, speech communications, and 
critical thinking. They were also asked to recommend three employers who hired their college 
graduates and three policymakers. 

From the 150 institutions we contacted, 50 academic vice-presidents returned incomplete 
nomination forms even though they agreed to participate. They usually recommended the 
requested faculty members. However, some administrators (at a variety of institutions) did not 
have any employer recommendations because they were not familiar with what particular 
companies hire their graduates. In these cases, we contacted the academic vice-president again 
and asked for a referral to another individual who would be knowledgeable about employers. 
At ten institutions, the directors of career counseling responded with more than the necessary 
number of employers who hire and recruit students from their own particular institution. 
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Table 1. Continued 



LIBERAL ARTS 


Atlantic Union.CoIIege 
Clinch Valley College 
Dakota State College 
Eastern Naziirene College 
Franklin and Marshall College 
Hartwick College 
Lincoln University 
Livingston University 
McKendree University 
Methodist College 
Metropolitan State University 


Miami University of Ohio 
Mount Ida College 
Neumann College 
Northwest Nazarene College 
Notre Dame College of Ohio 
Pacific Union College 
Rust College 

St. Mary's College of Maryland 
Teikyo Westmar University 
University of Findlay 
Whitman College 


TWO-YEAR 


Becker College 

Bergen Community College 

Bismarck State College 

Centralia College 

Columbia Basin College 

East Los Angeles Community College 

Erie Community College 

Finger Lakes Community College 

Guilford Technical Community College 

Harford Community College 


Howard Cominunity College 

John C. Calhoun Community College 

LaGuardia Community College (CUNY) 

Mohawk Valley Community College 

Mountain View College 

Penn State University, Schuylkill 

Pierce College 

Shelby State College 

Southwest Mississippi Community College 
Three Rivers Community College 



Table 2. Participating Institutions by Collapsed Carnegie Units 



Carnegie Type 


Sample 


Population 


Number 


Percentage 


Number 


Percentage 


Comprehensive 


20 


25.6 


573 


25.1 


Doctorate 


7 


9.0 


103 


4.5 


Liberal Arts 


22 


28.2 


463 


20.3 


Other Profession 


0 


0 


99 


4.3 


Reseiurch 


7 


9.0 


100 


4.4 


Two- Year College 


22 


28.2 


942 


41.3 



Some senior academic administrators were not willing to nominate policymakers. We phoned 
these administrators and learned that some of these leaders were afraid that potentially 
unfavorable results would be shared with their legislators or boards of trustees. They were also 
concerned about getting negative feedback from their own policymakers. These eight 
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administrators from particular institutions had completed nomination foms with the exception 
of policymakers. In these cases, we directly contacted policymakers in those particular states. 
We sent letters of invitation to the leaders of higher education coordinating boards and 
regional accrediting associatipns. Many of these policymakers agreed to participate. 

We included institutions in this study if they had at least completed all of the appropriate 
nominations for faculty. We wanted at least three nominations in each category so that we 
would have extra individuals to compensate for those people who would decline to participate. 



Table 3. Regional Accrediting Status 



Agency 


Sample 


Population 


Number 


Percentage 


Number 


Percentage 


New England Association of Schools 
and Colleges 


6 


7.7 


179 


7.9 


Middle States Association of Colleges and 
Universities 


20 


25.6 


403 


17.9 


North Central Association of Colleges and 
Universities 


20 


25.6 


765 


34.0 


Northwest Association of Schools and 
Colleges 


8 


10,3 


120 


5.3 


Southern Association of Colleges and Schools 


16 


20.5 


610 


27.1 


Western Association of Schools and Colleges 


8 


10.3 


176 


7.8 



Table 4. Control of Institutions 



Type 


Sample 


Population 


Number 


Percentage 


Number 


Percentage 


Public 


45 


57.7 


1384 


60.7 


Private, non-profit 


33 


42.3 


896 


39.3 



Table 5* Student Enrollments 



Size 


Sample 


Population 


Number 


Percentage 


Number 


Percentage 


Small 


29 


37.2 


1142 


50.1 


Large 


49 


62.8 


1138 


49.9 
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Therefore, we tried to get additional nominations from these senior administrators who 
nominated only one faculty member in each skill area. If we did not receive additional 
recommendations, these institutions were excluded from the study. 

For each nominated person, we contacted the individual by sending a personal letter indicating 
the specific senior administrator of a particular institution who had nominated them. This letter 
also included a description of the study and its purpose as well as the estimated amount of 
time to complete each survey. However, it was difficult to get nominated employers and 
policymakers to participate. Some individuals had too many commitments and cited their 
intense work schedule as the main reason for their inability to participate Therefore, we made 
personal phone calls to individuals in these two groups and were successful in recruiting 
additional participants. 

In each particular skill area, we had a group of faculty, employers, and policynudcers who 
completed our surveys (see Table 6). A total of 468 individuals were invited to complete the 
initial writing survey. The response rale from participants ranged from 35 percent for 
employers to 54 percent for faculty members. A very similar response rate was evident for the 
initial speech survey. A total of 478 individuals were invited to complete the critical thinking 
goals inventory. The response rate ranged from 21 percent for employers to 51 percent for 
faculty. In all cases, the response rate was highest for faculty and lowest for employers. 

In the second round of surveys, we invited the same individuals who had completed the initial 
surveys to participate again and re-evaluate their own judgements. The response rates for the 
second rounds of surveys remained high with a range of 64 percent of policymakers 
responding to the critical thinking instrument to 97 percent for employers responding to the 
same survey (see Table 7). The faculty constituted over one-half of the participants in the total 
respondent group with employers comprising nearly one-quarter and policymakers comprising 
nearly one-fifth of this group. 



K Data Collection 

All participants received the initial survey in the early part of the Fall 1993 semester or term. 
We designed three separate survey instruments in the following areas: writing, speech, and 
critical thinking. The instructions for each instrument were identical. We asked participants to 
indicate the importance of a variety of skills for college graduates. For the purpose of this 
study, we defined college graduates as both associate and baccalaureate degree recipients. The 
participants rated what skills students should have for work and citizenship. The scale ranged 
from extreme importance with a rating of **r' to no importance with a rating of "9." We asked 
participants to consider these skills in temis of the expectations they would have for individual 
students or employees within their current professional organization when they completed 
these surveys. For example, we asked faculty to consider that "if you are a college professor 
(at either a community college or a four-year institution), answer the items in terms of the 
skills you would expect from graduates of your own institution.'' 
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Table 6. Participants Responding to Initial Survey 



Round 1 Survey 


Type of 


Individuals 


Number of 


Response 


Percent 




Invited 


Respondents 


Rate Percent 




Writing 


raCUliy 




123 


54 


57.8 


Employers 


152 


53 


35 


24.9 




Policymakers 


88 


37 


42 


17.4 


Writing Total 










100.0 


Speech 


Faculty 


/Uo 






54.8 


Employers 


161 


55 


34 


26,2 




Policymakers 


91 


40 


44 


19,0 


Speech Total 




460 


210 




100.0 


Critical Thinking 


Faculty 


224 


113 


51 


62.8 




Employers 


165 


34 


21 


18.9 




Policymakers 


89 


33 


38 


18.3 


Critical Thinking Total 


478 


180 




100.0 


Overall Total Round I 




1406 


603 



Table 7. Participants Responding to Second Survey 



Round 2 Survey 


Type of 


Individuals 


Number of 


Response 


Percent 


Respondents 


Invited 


Respondents 


Rate Percent 




Writing 


Faculty 


123 


101 


82 


58.0 


Employers 


53 


43 


81 


24.7 




Policymakers 


37 


30 


81 


17.2 


Writing Total 




213 


174 




99.9 


Speech 


Faculty 


115 


90 


78 


53.6 


Employers 


55 


43 


78 


25.6 




Policymakers 


40 


35 


88 


20.8 


Speech Total 




210 


168 




100.0 


Critical Thinking 


Faculty 


113 


92 


81 


63.0 


Employers 


34 


33 


97 


22.6 




Policymakers 


33 


21 


64 


14.4 


Critical Thinking Total 




180 


146 




100.0 


Overall Total Round 2 




603 


488 
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If we did not receive the completed surveys within four weeks, we sent postcards to 
participants reminding them about tlie project and encouraging them to return the surveys in 
the next two weeks. When some individuals had still not responded, particularly for the 
employer and policymaker groups, we then called them as the final follow-up strategy. 

In the second survey, we included items with significant differences between the three 
respondent groups. For each item, we indicated the overall group mean. We asked each 
respondent to re-evaluate his or her position about the importance of these items. We also 
provided space for individuals to write their own comments outlining their reasons for 
disagreeing with the group mean. We followed the same procedures as described above to get 
participants to return their surveys. 

F. Data Analysis 

We used several stages in the statistical analysis of the data. These multiple steps were 
necessaiy since we did not have equal numbers of participants in the three groups of faculty, 
employers, and policymakers. The number of faculty who participated versus the employers 
and policymakers was extremely disparate. For the regular Analysis of Variance (ANOVA) 
model, it is assumed that the cell sizes of the three groups are balanced or equal and that there 
are equal variances among the three population groups. This model would give too much 
weight to the *'within group sums of squares'' for the faculty since these participants were the 
largest group. 

To compensate for these violations of the assumptions of the regular ANOVA model, we used 
the General Linear Model (GLM) to perform the analysis of variance. The GLM procedure 
(SAS, 1990) provides this analysis. This procedure was selected since it produces the 
necessary associated statistics required for this analysis. The first statistic was the least squares 
means which estimates the marginal means that would have been expected if the design had 
been balanced along with the probability that the expected means are equal. The second 
associated statistic was Tukey's studentized range test which calculates a confidence interval 
around the differences between the means. If the least square means probability was less than 
.05 and the lov^er confidence limit of the difference between two groups was close to zero or 
above, the question was included in the next step of the analysis. 

To produce the probabilities for the assumption of equal variances, unequal vai iances, and an 
associated F Test for the equality of variances, we used the T-Test next . Due to the small 
sample size of employer's and policymakers, we could not determine a reliable estimate of the 
population variance for each group. Therefore, the associated F Test for equal variances was 
not used. We employed the criteria that probabilities calculated for both equal and unequal 
variances had to be less than .05 for the differences between means to be judged significant. 
The specific items from the first questionnaire with these significant differences were included 
in the second sur\'ey. The same analysis procedures were followed for the second round of 
surveys. 
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We also conducted a reliability analysis to determine the interna! consistency of the 
iastrunients with respect to the individual sections as well as the instruments overall. High 
reliability coefficients greater than a = .65 demonstrate the consistency with which the sections 
are addressed by the respondents. A factor analysis was executed to demonstrate the construct 
validity of the instruments. We used this analysis to determine the validity of the instruments^ 
structures or divisions into specific sections. 
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IV. SURVEY RESULTS: IDENTIFICATION OF THE IMPORTANT SKILLS 



In this section, we present the results from the conimunication and critical thinking goals 
inventories. We highlight the areas of consensus among faculty, employers, and policymakers. 
We also report the areas where significant differences existed between the three respondent 
groups. We have incorporated the respondent comments to illustrate reasons for their different 
judgments. 

We report the results from the two rounds of surveys, including the total respondent group 
mean. In the areas of disagreement, we note the group average rating by each of the three 
respondent groups. The responses from the individual participants usually ranged from 
"extreme importance" (a rating of"!") to "no importance" (a rating of "9"). In section II, we 
describe more fully the data analysis procedures. 

A. Writing Skills 

A.i. Audience Awareness 

Most researchers agree that "audience awareness," or the ability to develop a representation of 
the potential readers of a text, is one of the most important cognitive skills for success in 
writing. Ideally when writers begin writing, they know who their audience will be and what 
relationship they hope to establish with them. Several empirical studies (Atlas, 1979; Beach & 
Anson, 1988; Berkenkotter, 1981; Flower 8c Hayes, i980b) have shown that audience 
awareness develops during the college years. Some of these studies (e.g.. Black, 1989; Rubin, 
1984) have characterized "audience awareness" in greater detail. Therefore, we included eight 
different items in this section of the survey. 

After the first round of Delphi surveys, faculty, employers, and policymakers agreed about the 
importance of three specific skills (see Table 8a). College graduates should be able to consider 
how ar audience will use a particular document, choose words that their audience can 
comprehend, and understand the relationship between the audience and the subject material. 
However, from the initial Delphi survey, there were five areas where respondents significantly 
differed (see Table 8a). Faculty mted all five skill areas as significantly more important than 
did either the employer or the policymaker. After the second round of surveys, the participants' 
responses reached a statistically significant agreement about the importance of an additional 
three skills. College graduates should be able to address audiences' different cultural and 
communication norms; understand the audiences' values, attitudes, goals and needs; and 
understand the relationship between audience and themselves (see Table 8b). 

After the second round of surveys, there were two specific skills where significant differences 
sfill existed among two of the groups (see Table 8b). Faculty continued to rate significantly 
higher than did employers the skills of addressing audiences whose backgrounds in the topic 
vary widely and defining multiple anficipated audiences. These skills are interrelated and 



31 



Table 8a* Analysis of Variance — Awareness and Knowledge of Audience 





Mean 


Mean 
Square 

Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Consider how an audience will use the document 


2,70 


4.53 


1.76 


.0790-^ 


Choose words that their audience can understand 


1.88 


1.32 


1.56 


.4298* 


Understand the relationship between the audience and the 
subject material 


2.05 


.07 


1.28 


.9495** 


Address audiences whose backgrounds in the topic 
vary widely 


2.91 


13.66 


2,26 


.0028* 


Address audiences whose cultural and communication 
norms may differ from those of the writer 


2.92 


13.96 


2.71 


.0065* 


Define their anticipated multiple audiences 


2,95 


21.87 


2.09 


.0001* 


Clearly understand their audiences' values, attitudes, 
goals, and needs 


2.58 


19.47 


1.57 


.0001* 


Understiind the relationship between the audience 
and themselves 


2.29 


5.94 


1.45 


.0179* 


SKILLS — ROUND 2 


Address audiences whose cultural and communication 
norms may differ from those of the writer 


2,94 


3.98 


1.52 


.0756* 


Clearly understand their audiences* values, attitudes, 
goals, and needc 


2.83 


3.02 


1.55 


.1457* 


Understand the relationship between the audience and 
themselves 


2.29 


2.17 


1.18 


.1615* 


Address audiences whose backgrounds in the topic 
vary widely 


2.90 


3.86 


1.44 


.0710* 


Define their anticipated multiple audiences 


2.92 


7.56 


1.35 


.0045* 


* Significant differences noted in TUKEY and Least Square Means 
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Table 8b. Disagreements about Awareness 
and Knowledge of Audience Between Respondent Groups 





Means 


Significance Level 


EMP 


FAC 

1 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 




Address audiences whose 
backgrounds in the topic vary widely 
Standard Deviation 


3.43 
1.59 


2.62 
1.41 


3.08 
1.67 


.00 1 2 


n.s. 


n.s. 


Address audiences whose cultural 
and communication norms may 
differ from those of the writer 

OlUflClUrCl L/zvlUHUH 


3.45 
1.81 


2.6-1 
1.43 


1 AO 

. .08 
1.87 


.l)0J2 


n.s. 


n.s. 


Define their anticipated multiple 
audiences 

Standard Deviation 


3.60 
1.68 


2.57 
1.40 


3.18 
1.20 


.0001 


n.s. 


.0101 


Clearly understand their audiences' 
values, attitudes, goals and needs 

Standard Deviation 


3.17 
1.44 


2.22 
1.16 


2.84 
1.24 


.0001 


n.s. 


.0080 


Understand the relationship 
between the audience and 
themselves 

Standard Deviation 


2.47 
1.30 


2.09 
1.19 


2.66 
1.07 


n.s. 


n.s. 


.0071 


SKILLS — ROUND 2 


Address audiences whose 
backgrounds in the topic vary 
widely 

Standard Deviation 


3.26 
1.38 


2.75 
1.17 


2.87 
1.01 


.0394 


n.s. 


n.s. 


Define their anticipated multiple 
audiences 

Standard Deviation 


3.42 
1.52 


2.71 
1.04 


2.90 
.96 


.0069 


n.s. 


n.s. 


EMP = Employers; FAC = Faculty; PM = Policymakers; n.s. = not significant 
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many respondents made the same remarks for both items as illustrations for their differences in 
perceptions. 

Part of the reason for these disagreements is that some employers view their own organizations 
as very homogeneous with little variation in terms of their organizational contexts for writing 
assignments. An employer states "as a police organization our requirements are somewhat 
unique. The vast majority of a police officer's writing consists of factual reporting with writing 
that is mostly devoid of creativity or emphasis. For legal reasons, reports cannot exceed the 
scope of the officer's field notes. This skill is not appropriate to police report writing, unless it 
is applied to the preparation by a few officers of lesson plans or other public presentations.'' 
Other employers state "entry level hires have limited involvement with varied audiences" and 
"I view this skill as something more advanced, it is of medium importance." Some employers 
view these skills as advanced abilities that new employees do not possess when they enter the 
work force. 

There are disagreements among some employers who believe it is critical to write a document 
so that individuals with different levels of knowledge about the topic will understand it. These 
employers may be individuals from more complex organizations who do have responsibilities 
to multiple audiences. The ability to address audiences whose background in the topic varies 
widely is "extremely important with the diversity in companies today'' states an employer. 
Another employer notes this skill is "very basic. Without it the graduate will not be educated 
to an acceptable level for professional growth." "In a cross-functional team environment, this 
[skill] is integral to share knowledge and exert influence. Our teams include individuals from 
finance, manufacturing, market research, product development, etc. Now [this is] more 
complex in global organizations" states an employer. 

Policymakers tended to agree that writing for diverse audiences is important. "Attorneys must 
be able to persuade audiences that the rule of law is critical to any society if it is to be a free 
society," states a policymaker. Another administrator remarks "I lead 100 clergy and this skill 
is critical to success." "[This is] a realistic demand for successful communication" states 
another policymaker. However, a respondent notes that this skill is "probably not important for 
a community college graduate." A policymaker notes that college graduates need to "expect [a] 
wide range of backgrounds (especially educational level) in [the] anticipated audience." 

The majority of faculty rated the ability to address and define multiple audiences as extremely 
important. Faculty teach a diverse grour^ of students with diverse career goals. "I rated all of 
the audience high because the entire purpose of writing is to reach someone, something that 
students often don't realize" states a professor. Another faculty member agrees that "many of 
our students begin their career here writing only for themselves. They need to learn to adapt 
[to] the audience's point of viewV "I rank all of these higher than the group mean. Audience is 
an important part of the writing process because the product isn't complete until it has been 
understood as accurately as possible. Understanding [the] audience allows the writer to choose 
a tone and voice that will not alienate the reader" states an instructor. In a similar manner, 
another professor emphasizes "If they [college graduates] cannot adopt their presentation to an 
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appropriate level, no communication will take place. Pitched too 'high' the audience will not 
understand; pitched too 'low' the audience may be insulted.'' 

Some faculty believe the ability to address and define multiple audiences is developed in the 
workplace once students enter a specific occupation. "An ability to write for audiences comes 
only with quite extensive knowledge of and experience with members of that audience. Often 
that knowledge and experience are gained in the workplace rather than in the classroom'' states 
a professor. Another faculty member concurs that this skill "is not easy to do in classroom 
settings, but [is] vital in business/organization settings." "The only antidote to super- 
specialization is the ability to speak to diverse audiences" notes another faculty member. A 
professor laments that "much poor writing and almost all poor public speaking is caused by 
the inability to gear material to the audience." A professor from the western part of the United 
States remarks that "sensitivity to audience [is] especially important in this area [of] southern 
California, and I think especially in writing." California has a more diverse population than 
many other states, and this particular comment illustrates how the geographic region where 
students and faculty work may intluence their responses. 

There were some faculty who did not believe the skill of addressing audiences whose 
background varies widely was extremely important. "The 'widely' is a high standard. 
Learning, for example, to make complex ideas simple may require more time. Understanding 
the wideness of cultural variation may require more life experience" emphasizes a professor. 
Another professor agrees "these are skills that, while desirable, require more experience than 
most college graduates will have." One professor notes "many graduates will never address an 
audience In speaking or writing whose background is unlike their own." 

Other faculty, employers, and policymakers noted the challenges of defining multiple 
audiences accurately. A common criticism of writing is that students do not learn to write for 
real audiences (Faigley et al., 1985). While this skill is viewed as extremely important by some 
respondents, others question the ability of college graduates to actually achieve it. As one 
faculty member notes "the act of defining an audience, as opposed to a relationship, is never 
really possible." However, an employer questions "If you don't know who your audience(s) 
are going to be, how do you know how to properly say what needs to be said?" 

In this section of the survey, faculty, employers, and policymakers disagreed about the 
importance of addressing audiences whose backgrounds in the topic vary widely and defining 
anticipated multiple audiences. However, they did agree that six specific skills were important 
for college graduates to develop. The most important skill, rated as extremely important, was 
the ability to choose words that audiences can understand followed by the ability to understand 
the relationship between the audience and the subject material and between the audience and 
themselves. The remaining skills they agreed upon were rated slightly lower in importance and 
included the ability to consider how an audience will use the document; understand audiences' 
values, attitudes, goals, and needs; and address audiences' different cultural and 
communication norms. 
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A.ii. Purpose for Writing 



Writers usually generate goals to help them reduce the number of constraints they must work 
with when they compose (Flower & Hayes, 1980b). Often writers develop goals from their 
long-term memory, but most goals are created by the writer in response to a specific situation 
(Faigley et al., 1985). Through their research. Flower and Hayes discovered that students 
create and revise goals throughout the composing process. These goals are often evaluated and 
revised in light of what has been written. Goals tend to develop as the written text progresses, 
and they interact with the text itself as well as with the situation for which the writing is being 
completed. 

Audience and purpose are often described as important elements of the rhetorical situation. 
Frameworks of the rhetorical situation (Bitzen 1968: Booth, 1963) usually include a persona 
(the image the writer wishes to project), an audience (the readers), and a subject (the 
information that the writer desires to impart) (Faigley et al., 1985). Therefore, approximately 
one-half of the items in this section of the survey indicated the relationship between the v/riten 
subject, and the audience relative to the purpose for writing. 

There were four areas of agreement from a total of seven regarding the importance of 
conveying the purpose for writing from the first round of surveys (see Table 9a). These skills 
included the ability to state purpose(s) to audiences, use vocabulary appropriate to subject and 
purpose(s), arrange words within sentences to fit the intended purpose(s) and audience(s), and 
to make appropriate use of creative techniques of humor and eloquence when approaching a 
writing task. 

There were three areas of statistically significant variations in responses between the three 
stakeholder groups. Faculty rated the awareness of multiple goals/purposes and the use of an 
appropriate tone of voice as more important than did the employers and the policymakers (see 
Table 9b). However, the policymakers rated significantly higher the ability for college 
graduates to draw on their own individual creativity and imagination to engage their audience 
than did employers. Their average ratings were also higher than faculty members' evaluations. 
After the second round of surveys, the three respondent groups agreed on the importance of 
creafivity. However, no consensus was reached in two areas (students should be aware of 
muUipIe purposes/goals; students should use an appropriate tone of voice) (see Table 9b). 

Faculty rated significantly higher than did employers and policymakers the ability to be aware 
of multiple purposes and goals. However, within ^he faculty group there were disagreements. 
Some faculty thought that multiple purposes was an extreme standard, and many times there 
would only be one purpose or goal that college graduates would attempt when they write. 
Faculty were concerned that students would become too '*self-conscious.'' Other faculty 
members disagreed. As one professor states '^effective writing is purposeful; most wrifing has 
more than one purpose [and] sometimes conflicting ones.'' Addifional faculty concur and 
remark this *'basic skill [is] needed,'' a **greater awareness of purpose will increase the 
likelihood of achieving it," and '^seeking this goal provides the key to successful expression 

36 



46 



Table 9a* Analysis of Variance — Purpose for Writing 





Mean 


Mean 
Square 

Model 


Mean 
Square 
EiTor 


Pr>F 


SKILLS — ROUND 1 


State their purpose(s) to their audiences 


2.04 


.17 


1.83 


.9097* 


Use vocabulary appropriate to their subject and purpose(s) 


1.81 


.63 


1.06 


.5546* 


Arrange words within sentences to fit the intended 
purpose(s) and audiences 


2.18 


1.51 


1.52 


.3710* 


Make appropriate use of creative techniques of humor 
and eloquence when approaching a writing task 


3.65 


7.61 


3.05 


.0845* 


Be aware of the multiple purposes and goals they are 
acting on when they write 


2.31 


11.04 


1.38 


.0004* 


Use an appropriate tone of voice 


2.28 


8.78 


1.79 


.0082* 


Draw on their individual creativity and imagination to 
engage their audience 


2.97 


13.86 


2.75 


.0073* 


SKILLS — ROUND 2 










Draw on their individual creativity and imagination to 
engage their audience 


2,72 


2.81 


1.41 


.1387* 


Be aware of the multiple purposes and goals they are 
acting on when they write 


2.17 


4.66 


0.75 


.0025* 


Use an appropriate tone of voice 


2.11 


2.53 


.98 


.0788* 


* Significant differences noted in TUKEY and Least Square Means 



and clarity of thought.'' A professor reiterated his point that this skill is extremely important 
since '^students have no clue why they write or how to use themselves and their feelings to 
form their writing.'' 

Policymakers cautioned that college graduates should be not "get paralysis from multiple 
purpose analysis," and some believe that "entry level workers [are] probably still too 
inexperienced" to achieve this skill. Some employers echoed this concern. As one employer 
notes "don't make communication too complex; keep it simple." However, other policymakers 
believe this skill is most important and that college graduates need to "know what they want to 
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Table 9b. Disagreements about Purpose for Writing Between Respondent Groups 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 




Be aware of the multiple purposes 
and goals they are acting on when 
they write 

Standard Deviation 


2.66 
1.22 


2.03 
1.09 


2.69 
1.36 


.0013 


n.s. 


.0078 


Use an appropriate tone of voice 

Standard Deviatu 


2.61 
m 1.48 


2.03 
1.31 


2.59 
M7 


.0129 


n.s. 


.0151 


Draw on their individual creativity 
and imagination to engage their 
audience 

Standan^ Deviation 


3.56 
1.78 


2.80 
1.60 


2.65 
1.66 


.0068 


.0114 


n.s. 


SKILLS — ROUND 2 


Be aware of the multiple purposes 
and goals they are acting on when 
they write 

Standard Deviation 


2.40 
.73 


1.99 
.73 


2.53 
1.33 


.0032 


n.s. 


.0397 


Use an appropriate tone of voice 

Standard Deviatii 


2.19 
m 1.03 


1.98 
.99 


2.43 
.93 


n.s. 


n.s. 


.0259 


EMP= Employers; FAC = Faculty; PM = Policymakers; n.s. = not significant 



There were also disagreements between faculty and policymaker groups about the importance 
of using an appropriate tone of voice. Some faculty thought that this skill was "too obvious to 
need special instruction" while others believed that this skill was "not as important as some 
other goals.'' One professor stated "if [students] can't do this, they're lost from the start." 
Another professor commented "unfortunately some writers have only one tone even after 
being educated." Faculty members cautioned "an inappropriate tone could destroy common 
ground. For example, authoritarian or informative in an egalitarian environment" and 
"inappropriate tones alienate readers." 

Policymakers shared similar concerns about the tone of voice. They remarked that the group 
"mean [is] much too low; graduates should be expected to have at least a rudimentary 
understanding and ability." Some policymakers disagree. They note the "tone is less important 
compared to clarity and purpose" and ''the substance is more crucial than the style." 
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Overall in this section of the survey, the three respondent groups disagreed about the 
importance of using an appropriate tone of voice and being aware of multiple purposes when 
writing. However, they did agree that five skills were critical for college graduates to achieve. 
The most important slcill rated as extremely important was the ability to use vocabulary 
appropriate to the subject and purpose(s) closely followed by stating purpose(s) to audiences, 
arranging words within sentences to fit the intended purpose(s) and audience(s), and drawing 
on individual creativity and imagination. Respondents rated the ability to make appropriate use 
of creative techniques of humor and eloquence when approaching a writing task as medium 
importance. 

A.iii. Pre-Writing Activities 

The pre-writing phase of composition usually involves planning activities that help writers 
prepare for their writing task. A planned writing task is an activity that has been thought out 
and designed prior to its expression. It requires preparation that often includes an analysis or 
creation process in order to generate ideas for writing (Camevale et al., 1990; Flower & 
Hayes, 1980a; Loacker et al., 1984). There was much less agreement in this area with only 
three items of consensus and eight items of disagreements from the first round of surveys. All 
stakeholder groups beheved that college graduates should be able to discuss their piece of 
writing with someone to clarify what they wish to say, research their subject, and identify 
problems to be solved that their topic suggests (see Table 10a). 

In five of the seven areas with disagreements, faculty rated certain skills significantly higher 
than did employers or policymakers (see Table 10b). These included the ability of college 
graduates to analyze their own experiences to provide ideas for writing; retrieve material from 
memory; plan the writing process using effective writing strategies and techniques; locate and 
present adequate supporting material; and focus and then narrow the plan by recognizing the 
rhetorical problem(s) they wish to solve. There were no significant differences between 
policymakers and employers relative to these six areas. However, in the seventh ability area of 
clarifying policy and position before writing, there were different perceptions. Both employers 
and policymakers rated this skill significantly higher than did the faculty. 

After the second round of surveys, the same seven specific skills still had statistically 
significant different ratings (see Table 10b). Again, similar trends emerged as those found from 
the first round of surveys. Two of the differences this time were relative to the skills of 
clarifying policy and position before writing as well as analyzing their own experiences for 
ideas. Policymakers agreed with faculty about the importance of these skill. However, 
employers in this second survey rated the clarification of one's view significantly higher than 
did the faculty members. 

College graduates should analyze their own experience to provide ideas for their writing. 
Faculty rated this particular skill significantly more important thai, did employers. A professor 
states "this helps to 'place' students and put themselves into it." Faculty members remark that 
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Table 10a. Analysis of Variance — Pre-writing Activities 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Discuss their piece of writing with someone to clarify what 
they wish to say 


3.33 


5.10 


3.53 


.2380 


Research their subject 

Identify problems to be solved that their topic suggests 


1.86 
2.47 


,56 
4.46 


1,17 
1.51 


.6233 
.0540 


Analyze their own experiences to provide ideas for 
their writing 


2.72 


32.86 


2,08 


,0001* 


Create ideas for their writing 


2,52 


15.95 


2.26 


,0011* 


Retrieve material from their memories to write 


3,17 


36,26 


2.59 


.0001* 


Plan writing processes, using effective writing strategies 
and techniques 


2.32 


16.02 


1.62 


,0001* 


Chu*ify their policy and position before writing 


3.23 


30.88 


4.43 


.0012* 


Locate and present adequate supporting material 


1.81 


5.33 


0.92 


,0035* 


Focus and then narrow their plan by recognizing the 
rhetorical problem(s) they wish to solve 


2.76 


15.44 


2.06 


.0007* 


SKILLS — ROUND 2 


Analyze their own experiences to provide ideas for 
their writing 


2.79 


4.99 


1.22 


.0183* 


Create ideas for their writing 


2.73 


6.73 


l.ll 


.0029* 


Retrieve material from their memories to write 


3.02 


5.34 


1.47 


.0286* 


Plan writing processes, using effective writing strategies 
and techniques 


2,02 


5.14 


0.60 


,0003* 


Clarify their policy and position before writing 


3.05 


7.43 


2,22 


.0377* 


Locate and present adequate supporting material 


1,83 


3.58 


0.51 


.0011* 


Focus and then narrow their plan by recognizing* the 
rhetorical problem(s) they wish to solve 


2.76 


5.58 


1.06 


.0061* 


* Significant differences noted in TUKEY and Least Square Means 



40 



Table 10b. Disagreements about Pre-writing Activities Between Respondent Groups 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMr/F:Vl 


hAL/riVl 


SKILLS — RODINL) I 








Analyze their own experiences to 
provide ideas for their writing 

Standard Deviation 


3.48 
1.62 


2.25 
1.31 


3.08 
1,56 


.0001 


n.s. 


.0039 


Create ideas for their writing 

Standard Deviation 


3.10 
1.59 


2.21 
1.51 


2.63 
1.35 


.0005 


n,s. 


n.s. 


Retrieve material from their 
memories to write 

Standard Deviation 


3.79 
1.72 


2.66 
1.55 


3.85 
1.63 


.0001 


n,s. 


.0002 


Plan writing processes, using 
effective writing strategies and 
techniques 

Standard Deviation 


2.88 
1.51 


2.00 
1,15 


2,50 
1.24 


.0002 


n.s. 


,0278 


Clarify their policy and position 
before writing 

Standard Deviation 


2.81 
1,83 


3.69 
2.40 


2.43 
1.36 


,0076 


n.s. 


.0001 


Locate and present adequate 
supporting material 

Standard Deviation 


2.07 
1.02 


1.61 
.911 


2.03 
1,00 


,0044 


n.s. 


.0230 


Focus and then narrow their plan by 
recognizing the rhetorical 
problem(s) they wish to solve 

Standard Deviation 


3.28 
1.56 


2.44 
1.44 


3,03 
1.20 


.0008 


n.s. 


,0137 


SKILLS — KUUND Z 














Analyze their own experiences 
to provide ideas for their writing 

otanaara ueMaiion 


3.19 
1.G3 


2,61 
1.15 


2.80 
1.03 


.0040 


n.s. 


n.s. 


Create ideas for their writing 

Standard Deviation 


3,05 
1.11 


2,49 
1.01 


3.07 
1.11 


.0062 


n.s. 


.0146 


Retrieve material from their 
memories to write 

Standard Deviatiot 


3.23 
1.13 


2.81 
1.24 


3.40 
1.25 


.0496 


n.s. 


.0275 


continued on next page 
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Table 10b. Continued 





Means 


Significance Level 


BMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


Plan writing processes, using 
effective writing strategies and 
techniques 

Standard Deviation 


2.28 
.78 


1.81 
.71 


2.33 
.92 


.0013 


n.s. 


.0065 


Chirify their policy and position 
before writing 

Standard Deviation 


2.67 
1.08 


3.30 
1.69 


2.76 
1.27 


.0090 


n.s. 


n.s. 


Locate and present adequate 
supporting material 

Standard Deviation 


2.05 
.69 


1.66 
.57 


2.10 
1.08 


.0018 


n.s. 


.0399 


Focus and then narrow their plan 
by recognizing th^ rhetorical 
problem{s) they wish to solve 

Standard Deviation 


3.05 
.90 


2.54 
1.04 


3.07 
1.16 


.0040 


n.s. 


.0321 


EVIP = Employers; FAC = Faculty; PM = Policymakers; n.s. = not significant 



"to analyze and write about one's own experiences is to understand it better" and **persons 
who do not [have this skill] cannot relate to the experiences of others " Faculty also emphasize 
that "a necessary first step to the more detached critical modes [is] a need to understand our 
own conceptual frameworks first " and "students need to see the value in reflecting and 
preparing to write and the role of thinking as a vital part of writing " *A1I learning is 
experiential, if they can't analyze their experience, they can*t analyze*' stresses another 
professor. "How else do you synthesize language and experience to say something useful" 
questions a faculty member. However, some professors disagree. As one instructor comments 
"our emphasis is on teaching students to analyze other people's writing." "There is often little 
carryover from personal experiences to professional contexts for writing" remarks a professor. 

Employers believed that analyzing one's own experience to provide ideas for writing is not 
critical within their organizational contexts, and they rated it lower than did the faculty. This 
skill is **not as important because most experiences are learned on the job" notes an employer. 
Another individual states "since their experience is limited, they need to look well beyond 
themselves." In another company, "reports and writing are reviewed [since] prior experience 
may not exist in areas they write about" states a supervisor. 

Faculty believed that college graduates should be able to create ideas for their writing 
especially if a goal is to prepare future leaders in our society. They rated this skill significantly 
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higher than did employers and policymakers. Faculty valued ^^originality" and the generation 
of new ideas. These skills were critical for academic success according to faculty. They stated 
"generating ideas is central to most, if not ail» writing tasks " "writing should be, in part, self- 
expression," this is "at the heart of what writing is all about," "brainstorming is important in 
the pre-writing stage," and "creative thoughts are not just for artists and philosophers. They 
[college graduates] need to learn the generative capacities of their own language." However, a 
skeptical professor states "create? If you can understand and use an idea, that's the most that 
can be hoped for [from college graduates]." 

Both employers and policymakers did not share the faculty values. They were troubled by the 
word "create" in this skill. They believed that creativity is not necessary for some jobs and that 
some college graduates are not capable of generating new ideas. As a policymaker notes "some 
people are just not creative and/or their memories are not fountains of pertinence.'' Another 
administrator states "the ideas are usually given to the neophyte who must write, i.e. flesh out 
and discuss the idea." A faculty member notes the differences in organizational contexts and 
states '*these skills would be important only if graduates were engaged in jobs that required 
informed autobiographical writing, surely a small percentage would fit into this situation." 

Faculty members rated the ability to retrieve material from memories to write significantly 
higher than did employers and policymakers. This was another skill that professors value and 
hope college graduates achieve. As a composition professor comments, this skill is "common 
in early composition classes [but I am] not sure it is useful later/' Another professor concurs 
and states college graduates "should and can do [this] in class, [but] it is harder to learn on the 
job. A writer's memory is important to effective writing." "Students need to trust 'invention' 
capacities in addition to finding authorities as sources" and "memory is what allows synthesis" 
states the faculty. Another professor laments "often we have no precedents in memory for the 
fast-paced worlds in which we write." 

Both employers and policymakers rated lower the use of memories in writing compared to the 
faculty ratings. As a policymaker notes "although im.portant in some areas, most policy writing 
does not require graduates to retrieve material from their own memories. Policy writing 
demands [an] ability to abstract, integrate and conceptualize well." An employer shares a 
similar view and stresses "technical papers are written from facts — verifiable facts." "Memory 
is but one source" and "it depends on the type of writing being done. Memory is not required 
for some writing" according to several policymakers. Another policymaker comments 
"memory would be involved in pre-writing for brainstorming purposes. [However,] 
cooperative groups could enter here in place of individual memory." 

Similar disagreements '\rise relative to whether college graduates should be able to plan 
writing processes using effective writing strategies and techniques. Faculty rated this skill 
significantly higher than did employers and policymakers. Employers tended to not offer 
reasons for their different perceptions on this item. One reason for this lack of comments is 
that this item may reflect academic concerns which is an area that faculty can readily identify 
as being extremely important. Faculty members comment "knowing how to plan writing is a 

43 



ERLC 



53 



kind of problem-solving skill that can be applied to multiple purposes;' '^without planning, 
even the best ideas go nowhere^ and ^'knowing one's writing process ensures greater success 
because one can then allot the needed time to each stage of the process. Without this 
knowledge, planning and revising is apt to be incomplete, resulting in poor communication." 
Another faculty member cautions '*plan by all means, but be flexible for the organization of 
writing." A faculty member believes that this skill is "more pertinent than sensitivity to the 
audience " Another professor remarks *1 do teach that this is essential. Graduates ought to be 
equipped with a variety of strategies and techniques to choose from to suit any future writing 
purpose.'' However, two professors disagree and note "planning is probably less important than 
being able to do'' and '*this is too canned and formulaic." Several policymakers are concerned 
thai this particular skill can't be expected from college graduates who lack work experience. 
Only through experiences in their occupations can college graduates acquire the maturity 
necessary to fully develop this particular skill according to these administrators. 

Faculty believe it is extremely important for college graduates to locate and present adequate 
supporting material. They rated this skill significantly higher than employers and 
policymakers. Again, employers offered few comments to support their different perceptions. 
Faculty emphasized this skill is panicularly important in developing persuasive or 
argumentative discourse and in trying to persuade others of different viewpoints. Faculty 
comment don't see how any writing is meaningful without evidence. Mere generalizations 
are simply not enough," and this is a '*key to a good essay." Policymakers and faculty noted 
that this is a skill where cun*ent college graduates are weak. *Too few students understand 
evidence and how to use it" states a professor 

Significant differences emerged relative to college graduates' abilities to focus and then 
narrow their plan by recognizing the rhetorical problem(s) they wish to solve. Faculty rated 
tliis skill significantly higher than did employers and policymakers. This particular survey item 
was recommended by several writing faculty advisory board members. However, some 
employers and policymakers in the study did not understand the term "rhetorical problem." 

Faculty tended to perceive this skill as basic while policymakers viewed it as too advanced a 
skill for college graduates to achieve. This skill is "essential for any writing task to be 
successful" states a faculty member Another professor believes it is a "continuation of 
^planning' and just as important." Policymakers remark "this is rather difficult for a young 
person" and **much too advanced a skill... [College graduates] need maturity /experience and 
further education" in order to achieve this skill. 

The ability of college graduates to clarify their policy and position before writing generated 
the largest number of comments from the respondent groups. Employers rated this skill 
significantly higher than did the faculty. Some faculty believed that this clarification occurs 
while students write and revise their papers, and they objected to the need to do this before the 
writing begins. However, other faculty believed it is extremely important to clarify one's 
position before writing. "Basic policy and position must be defined as a precondition for any 
kind of writing in context" states a professor. This skill "in technical writing is very important" 
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comments another faculty member. A faculty member remarks "although it is true that writers 
discover meaning as they write, it's also true that they need a clear sense of direction before 
they begin drafting." An employer concurs and comments "before you start writing, you must 
know what your purpose is." Another employer offers a practical example that "often times 
[when] requesting additional funding [you] need to specify reasons." Policymakers stress 
"clarity of thought is essential, not only about the topic but about what the graduates thinks 
about the topic. The writer's attitude will be reflected in the work," and it is "difficult to write 
clearly without doing so." 

Some faculty disagree. They remark "clarification can wait for revision"; "it is only through 
writing that one can achieve clarity of thought," "there is seldom a need to do so before 
writing." Another professor strongly concludes "this is exactly what students are too often 
taught to do, and this practice both forestalls beginning to write and produces shallow, 
perfunctory writing that is short on thinking. Students need to write and revise in order to 
discover their policy or position." 

In pre-writing activities, there were many disagreements about the importance of certain skills 
including analyzing experiences to provide ideas, creating ideas for writing, retrieving material 
from memory, planning writing processes, clarifying position and policy before writing, 
locating and presenting supporting material, and recognizing rhetorical problems and then 
focusing and narrowing the writing plan. The respondent groups agreed only that three skills 
were important. College graduates' abilities to research their subject was rated the most 
important in this section followed by the ability to identify problems to be solved, and to 
discuss a piece of writing with someone to clarify what to say. 

A.iv. Organizing 

An important skill cited in many studies (Anderson, 1985; Cullen et al., 1987; Davis & 
Stohrer, 1989; Faigley et al., 1981; Goswami et al, 1981; HaswelL 1984; Loacker et al.. 1984; 
Storms, 1983; White & Polin, 1986; and Witte et al., 1982) is the ability to clearly organize 
and structure a document. Haswell (1984) concluded that the essays of upper-division college 
students provided more evidence of logical organization of ideas and had clearer connections 
between paragraphs than lower division students. Furthermore, White's and Polin's (1986) 
survey of the California State University instructors indicated that the ability "to select, 
organize, and present details to support a controlling idea" was important as well as the ability 
to "use appropriate organization and paragraphing." The establishment of an order for the 
writing task involves the consideration of the needs of the subject matter and the potential 
readers especially for the more experienced writers (Faigley et al., 1985). 

There were 18 different items in this section of the survey. From the first round of responses, 
there was a consensus on six of these items (see Table 1 la). These areas included the ability to 
organize material for more than one audience; include clear statements of the main ideas; 
demonstrate methods of organization to audience(s) by using informative headings; write 
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Table 11a. Analysis of Variance — Organizing 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Organize the material for more than one audience 


3.34 


1.86 


2.56 


.4857 


Include clear statements ot the main ideas 


1.67 


2.09 


.99 


.1246 


Demonstrate their method of organization to their 
audience(s) by using informative headings 


3.50 


4.90 


3.39 


.2386 


Write informative headings that match their audiences' 
questions 


4.01 


5.42 


3.35 


.2000 


Maintain coherence within sentences 


1.57 


1.38 


.60 


.1014 


Maintain coherence among sentences, paragraphs, and 
sections of a piece of writing 


1.59 


1.68 


.67 


.0826 


Develop patterns of organization for their ideas 


1.88 


3.15 


1.00 


.0453* 


Use knowledge of t.ieir subject matter to shape a text 


2.16 


8.26 


1.00 


.0004* 


Use knowledge of potential audience expectations and 
values to shape a text 


2.52 


11.43 


1.65 


.0012* 


Create and use an organizational plan 


2.90 


8.26 


3.07 


.0700* 


Select, organize, and present details to support a main idea 


1.71 


10.98 


0.98 


.0001* 


Organize their writing in order to emphasize the most 
important ideas and information within sentences and 
larger units such as paragraphs. 


1.86 


4.47 


1.08 


.0174* 


Cluster similar ideas 


2.40 


9.75 


1.81 


.0052* 


Provide a context for the document in the introduction 


2.62 


13.48 


2.20 


.0026* 


Maintain connections that link key points in an argument 
with multiple points 


2.02 


8.89 


1.53 


.0035* 


Set up signposts such as tables of contents, indexes, 
and side tabs 


4.16 


12.02 


3.87 


.0467* 


Demonstrate patterns of reasoning in their writing 


2.19 


14.86 


1.78 


.0003* 


continued on next page 
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Table 11a. Continued 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


Move between more abstract and more specific levels 
of argument 


2.49 


50.82 


1.85 


.0001* 


SKILLS — ROUND 2 


Develop patterns of organization for their ideas 


1.95 


.49 


.51 


.3856 


Use knowledge of potential audience expectations and 
values to shape a text 


2.69 


2.01 


.95 


.1250 


Create and use an organizational plan 


2.66 


.10 


1.29 


9236 


Organize their writing in order to emphasize the most 
important ideas and information within sentences and 
larger units such as paragraphs. 


1.92 


.86 


.41 


.1284 


Cluster similar ideas 


2.08 


.94 


.55 


.1866 


Provide a context for the document in the introduction 


2.42 


1.79 


.86 


.1292 


Set up signposts such as tables of contents, indexes, 
and side tabs 


4.19 


1.08 


2.02 


.5865 


r emonstrate patterns of reasoning in their writing 


2.16 


.39 


.88 


.6457 


Use knowledge of their subject matter to shape a text 


2.08 


3.18 


0.60 


.0060* 


Select, organize, and present details to support a main idea 


1.75 


2.00 


0.38 


.0059* 


Maintain connections that link key points in an argument 
with multiple points 


2.01 


4.58 


0.50 


.0002* 


Move between more abstract and more specific levels 
of argument 


2.75 


10.41 


1.19 


.0002* 


* Significant differences noted in TUKEY and Least Square Means 



47 

5? 

ERIC 



informative headings that match audiences' questions; maintain coherence within sentences; 
and maintain coherence among sentences, paragraphs, and sections of a piece of writing. 

By the second round of surveys, the three respondent groups reached an agreement about the 
importance of an additional eight skills for a total of 14 items (see Table 1 la). The three 
respondent groups agreed that college graduates should be able to develop patterns of 
organization for their ideas; use knowledge of potential audience expectations and values to 
shape a text; create and use an organizational plan; organize writing in order to emphasize the 
most important ideas and information within sentences and in larger units such as paragraphs; 
cluster similar ideas; provide a context for the document in the introduction; set up signposts such 
as table of contents, indexes, and side tabs; and demonstrate patterns of reasoning in their writing. 

After the second round of surveys, respondents disagreed about the importance of four skills 
(see Table I lb). In all cases, the faculty rated these skills significantly higher than did either 
employers or policymakers. Faculty believed that college graduates should use the knowledge 
of their subject matter to shape a text. In general, faculty comment that -'knowledge of a 
subject is vital to writing cogently," "essential goal of instruction in higher education," "a 
precondition for successful writing," and "ideally [this is a goal] but most of us have trouble 
doing that." Policymakers disagreed with this viewpoint. They state this "may not be a needed 
source of knowledge." "while important, this is not the prime factor," and "this is more than 
many college graduates know." Employers tended to concur with these policymaker views. As 
one employer notes "we have a number of entry-level staff members who must do this 
independendy, prior to a final review. We could not spend the time with each person helping in 
these areas." 

Another .skill where faculty significantly disagreed with the other two respondent groups was 
the importance of college graduates' abilities to select, organize, and present details to support 
a main idea. Faculty state "college teachers of writing rate this at the top of the list, even when 
they don't agree on other writing matters," "this is evidence of critical thinking abilities," 
"heart of coherence in an essay," and "no main idea is convincing without effective dettiils. * 
unless the main idea was self-evident in the first place." Some employers and policymakers 
viewed this skill as mK)rt applicable to writers who write longer documents which are often not 
necessary in the workplace. As an employer notes "these infer rather long written 
communications, sometimes the best communications are short, succinct statements." A 
policymaker places an emphasis on the key word 'select' and stresses that "too many details 
spoil the broth." Faculty believe this is a critical skill while others question whether college 
graduates need this skill for the workplace. 

Faculty rated significantly higher than did employers and policymakers the ability to maintain 
connections that link key points in an argument with multiple points. A professor states 
"connecting central points with others is important to the development and persuasiveness of 
writing. However, some faculty disagree. As one professor comments "this while important, is 
less so, only if they're attempting graduate school would they really need this in-depth." 
Policymakers and employers believe this skill is too advanced for college graduates. 
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Table lib. Disagreements about Organizing Activities Between Respondent Groups 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND I 


Develop patterns of organization 
for their ideas 

Standard Deviation 


2.07 
.95 


1.73 
.99 


2.08 
1.09 


.0303 


n.s. 


n.s. 


Use knowledge of their subject 
matter to shape a text 

Standard Deviation 


2.47 
1.06 


1.92 
.94 


2.48 
1.09 


.0011 


n.s. 


.0051 


Use knowledge of potential 
audience expectations and values 
to shape a text 

Standard Deviation 


2.86 
1.49 


2.23 
1. 10 


2.90 
1.46 


.0051 


n.s. 


.0103 


Create and use an organizational plan 
Standard Deviation 


2.78 
1.64 


3.11 
1.90 


2.40 
1.37 


n.s. 


n.s. 


.0115 


Select, organize, and present details 
to support a main idea 

Standard Deviation 


2.16 
1.35 


1.44 
.77 


1.90 
.98 


.0003 


n.s. 


.0089 


Organize their writing in order to 
emphasize the most important ideas 
and information within sentences 
and larger units such as paragraphs 
Standard Deviation 


2.19 
1.08 


1.71 
1.02 


1.85 
1.05 


.0059 


n.s. 


n.s. 


Cluster similar ideas 

Standard Deviation 


2.60 
1.47 


1.98 
1.29 


2.53 
1.32 


.0065 


n.s. 


.0247 


Provide a context for the document 
in the introduction 

Standard Deviation 


3.19 
1.42 


2.37 
1.50 


2.55 
1.53 


.0005 


0.0398 


n.s. 


continued on next page 



Employers and policymakers rated significantly lower than did faculty the ability to move 
between more abstract and more specific levels of argument. A professor notes "abstraction 
without specifics often baffles an audience. All specifics and no ability to synthesize leaves an 
audience guessing.'' Again, some employers and policymakers believe this skill may be too 
advanced for college graduates. 
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Tabic lib. Continued 





Means 


Significance Level 




C\ ID 


PAP 


PM 


FMP/FAC 


EMP/PM 


FAC/PM 


Maintain connections that link 
key points in an Jirgument with 
multiple points 

Standa-d Deviation 


2.48 

1.47 


1 O 1 

1.81 
1.18 


I .Vo 

1.02 




W.\/*tD*t 




Set up signposts such as tables 
of contents, indexes, and side tabs 

Standard Deviation 


4.02 
1.99 


4.41 
1.94 


3.55 
2.02 


n.s. 


n.s. 




Demonstrate patterns of reasoning 
in their writing 

Stan da nl Devi at ion 


2.74 
K58 


1.89 
1.29 


2.33 
1.05 


.0005 


n.s. 


.0330 


Move between more abstract and 
more specific levels of argument 

Standard Deviation 


3.47 
1.69 


L91 
1.16 


2.85 
1.41 


.0001 


0.0511 


.0003 


SKILLS — ROUND 2 




Use knowledge of their subject 
matter to shape a text 

Standard Deviation 


2.26 
.90 


1.92 
.60 


2.37 
1.07 


.0290 


n.s. 


.0351 


Select, organize, and present 
details to support a main idea 

Standard Deviation 


1.86 
.64 


L62 
-53 


2.00 
.83 


.0361 


n.s. 


.0246 


Maintain connections that link key 
points In an argument with 
multiple points 

Standard Deviation 


2.30 
.83 


1.81 
.53 


2.23 
1.01 


.0007 


n.s. 


.0336 


Move between more abstract and 
more specific levels of an argument 
Standard Deviation 


3.26 
1.20 


K67 
1.10 


2.97 
.89 


.0004 


n.s. 


.0123 


EMP = Employers: FAC = Faculty: PM = Policymakers; n.s. 


= not significant 







The three respondent groups agreed about the importance of 14 different skills. The ability to 
maintain coherence within and among sentences, paragraphs, and sections of a piece of 
writing as well as the inclusion of clear statements about main ideas were rated as the most 
important skills. The ability to set up signposts and write informative headings that match 
audience's questions were rated medium importance. There were four skills that faculty 
consistently rated of higher importance than did the employers or policymakers. These areas 
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included linking key points in an argument; moving between abstract and specific levels of 
argument; selecting, organizing, and presenting details to support a main idea; and using 
knowledge of subject matter to shape a text. 



A.v. Drafting 

Most writers complete a first draft of their written work. Flower and Hayes (1985), as well as 
Faigley et al. (1985) note that the actual process of drafting by skilled writers involves the 
generation of ideas based on goals involving the relationship between the reader and the 
writer. Less-skilled writers are more likely to generate ideas using simple remembered facts 
about the topic. 

In this section of the survey, the three respondent groups reached a consensus on five items 
after the initial round of feedback (see Table 12a). These abilities included avoiding common 
grammatical errors of standard written English, quoting accurately, establishing and 
maintaining a focus, writing effective introductions and conclusions, and writing effectively 
under pressure and meeting deadlines. 

They also rated lower than did faculty the skill of refining the notion of audience(s) as they 
write. Faculty remarked **audience analysis is critical to success;" **for anything more 
complicated than filling in forms, this is essential;" **not considering audience and purpose 
lessens the probability that the intended message will be received;" ^^globalization and 
diversity requires this skill;" **audience is the most important context for any form of 
communication " **an inauthentic voice can undermine writing;" and "a clear and defined 
persona provides context and an orientation for a reader." However, some policymakers and 
employers object to the emphasis on the audience and believe that content needs to be stressed 
more than anything else. Others think this skill can only be developed in mature writers who 
have gained experience in their work setting. Some employers believe this skill should only 
occur once and that no re-work be involved in writing. They also believe that entry level 
employees should not be expected to possess these skills. Even some faculty disagree. As one 
professor notes "I resent the notion that a writer is like an actor, always adapting him/herself to 
a myriad of environments." "Choice implies a degree of mastery, ^consider' doesn't. Mastery 
is a good goal, but can be inhibiting and is in any case an unreachable ideal. How can any of 
us *choose' the voice we write in?" and ''Is audience awareness even taught? Do college 
graduates really write to differing ones?" question two professors. 

Faculty, employers, and policymakers disagreed about eight specific skills (see Table 12b). 
However, after the second round of surveys, participants agreed about three additional items 
including the ability to make general and specific revisions; to move between reading and revising 
of their drafts to emphasize key points; and to refine norion of audience while writing. This meant 
there were five items remaining where disagreements still existed (see Table 12b). Again, in most 
cases the faculty rated these skills significantly higher than did employers or the policymakers. 
There were no significant differences in the ratings between employers and policymakers. 
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Table 12a. Analysis of Variance — Drafting 





Mean 


Mean 
Square 

Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROLND 1 


Avoid common ^rammuticul errors of standard 
written English 


1.60 


1.56 


1.01 


.2170 


Quote accurately 


1.56 


2.18 


1.20 


.1642 


PctoWlich 'mrl m'llnt'Hn '1 rf^PI!<C 
ChlaUlI^n ullij Inallllulll J. lUCUo 


1.51 


1.33 


.55 


.0916 


A\/ritA *>ffV»rtivf> i ntrnrliiptinn^^ nnd conclusions 


1.S3 


1.01 


1.29 


.4632 


Writ/a f^ffi^rtivpi V imH**r nr'=**vv:nrp ;ind meet deadlines 


2.41 


.75 


1.90 


.6733 


vvriic ciiuiis ui iiicii wuiiv 


1.84 


17.69 


1.27 


.0001* 


Make general and specific revisions while they write 
their drafts 


2.12 


20.74 
7.18 


1.94 


.0001* 


Develop paragraphs 


1.63 


1.12 


.0020* 


Move between reading and revising of their drafts to 
emphasize key points 


2.10 


9.04 


2,01 


.0122* 


Develop their chosen topic 


1.57 


10.47 


0.89 


.0001* 


Refine goals as they write 


2.16 


26.92 


1.44 


.0001* 


Refine the notion of audience(s) as they write 


2.73 


24.04 


2.08 


.0001* 


Consider audience and purpose to shape their voice, 
choose a voice to write in, and choose language, sentence 
structure and content to create that voice 


2.43 


12.06 


2.3^ 


.0065* 


SKILLS — ROUND 2 




Make general and specific revisions while they write 
their drafts 


2.20 


1.62 


1.45 


.3307 


Move between reading and revising of their drafts to 
emphasize key points 


2.12 


1.40 


.73 


.1479 


Refine the notion of audienceCs) as they write 


3.08 


2.96 


1.53 


.1472 


Write drafts of their work 


1.86 


3.29 


0.61 


.0053* 


continued on next page 
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Table 12a. Continued 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


Develop paragraphs 


1.73 


1.85 


0.44 


.0158* 


Develop their chosen topic 


1.76 


1.55 


0.45 


.0333* 


Refitie goals as they write 


2.05 


3.69 


0.74 


.0078* 


Consider audience and purpose to shape their voice» 
choose a voice to write in» and choose language, 
sentence structure and content to create that voice 


2.59 


5.75 


1.47 


.0216* 


* Significant differences noted in TUKEY and Least Square Means 



Faculty believed that it was extremely important for college graduates to write drafts of their 
work. They remark **drafting is important because it helps develop more complex thinking," 
"people attend college to learn and develop these basic skills" and [this skill] "is extremely 
important because it implies that the writer embraces revision." Another professor comments, 
"I'm very much convinced of the usefulness of the ^process approach'— teaching students to 
draft and revise is essential " 

Employers and policymakers rated this skill significantly lower than did faculty. One employer 
states that his employees usually write a narrative directly from their own notes. These two 
respondent groups believe college graduates may not need to write several versions of their 
work. Other respondents including some faculty imply that the need to write several drafts is 
associated with students who have weaker writing skills. As one professor stresses "some 
students learn to write well with minimal revision — more of a focus for weak students." 
"Some people don't need several drafts" states a policymaker. A faculty member suggests 
students "should be able to recognize when a draft is helpful and when it is not.'^ Another 
policymaker comments "drafts are initial in collaborative work settings, and they often have to 
be done quickly." This particular statement may capture part of the reason for differences 
regarding this skill. In college, students usually have many opportunities to write and create 
different drafts. However, once they enter the workforce, time constraints or company 
schedules may impose upon employees a real demand to accomplish writing quickly with little 
opportunity for the generation of multiple draft versions. However, the faculty themselves 
disagree about the importance of this skill. 

Faculty rated significantly higher than did policymakers the ability of college graduates to 
develop paragraphs. Most faculty believe these are fundamental skills. They state 
"development as long as it is not empty amplification is essential," "draft paragraphs may 
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Table 12b. Disagreements about Drafting Activities Between Respondent Groups 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


PMP/PM 
ciVlr/riv! 


FAr/PM 


SKILLS — ROUND 1 






Write drafts of their work 

Standard Deviation 


2.38 
1.18 


1.49 
.94 


2.13 
1.50 


.0001 


n.s. 


.0150 


Make general and specific revisions 
while they write their drafts 

Standard Deviation 


2.71 
1.27 


1.74 
1.3i 


2.43 
1.77 


.0001 


n.s. 


.0280 


Develop paragraphs 

Standard Deviation 


1.90 
1.07 


1.41 
.85 


1.95 
1.53 


.0029 


n.s. 


.0379 


Move between reading and revising 
of their drafts to emphasize key points 
Standard Deviation 


2.50 
1.49 


1.85 
1.29 


2.28 
1.67 


.0053 


n.s. 


n.s. 


Develop their chosen topic 

Standard Deviation 


2.07 
1.41 


1.34 
.67 


1,53 
.78 


.0004 


.0167 


n.s. 


Refine goals as they write 

Standard Deviation 


2.88 
1.55 


1.74 
.95 


2.43 
1.32 


.0001 


n.s. 


.0037 


Refine the notion of audience(s) as 
they write 

Standard Deviation 


3.37 
1.79 


2.33 
1.29 


3.08 
1.32 


.0002 


n.s. 


.0029 


Consider audience and purpose to 
shape their voice, choose a voice to 
write In, and choose language, 
sentence structure and content to 
create that voice 

Standard Deviation 


2.83 
1.83 


2.14 
1.41 


2.77 
1.40 


.0128 


n.s. 




SKILLS — ROUND 2 






Write drafts of their work 

Standard Deviatior 


2.09 
I .87 


1.69 
.68 


2.07 
.94 


.0082 


n.s. 


.0474 


Develop paragraphs 

Standard Deviatior 


1.74 
I .73 


1.63 
.59 


2.03 
.76 


n.s. 


n.s. 


.0119 


Develop their chosen topic 

Standard Deviatior 


1.95 
t .84 


1.65 
.61 


1.87 
.57 


.0389 


n.s. 


n.s. 


continued on next page 
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Table 12b. Continued 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/F.AC 


EMP/PM 


FAC/PM 


Refine goals as they write 

Standard Deviation 


2.40 
1.33 


1.90 
.64 


2.03 
.61 


.0241 


n.s. 


n.s. 


Consider audience and purpose to 
shape their voice, choose a voice to 
write in, and choose hinguage, 
sentence stnicture and content to 
create that voice 

Standard Deviation 


2.84 
1.43 


2.38 
1.11 


2.97 
1.19 


n.s. 


n.s. 


.01941 


EMP = Employers; FAC = Faculty; PM = Policymakers; n.s. 


= not significant 







change, but even in drafting they signal new ideas," "necessary to show control of an 
organized un'iC and "the audience needs a sense of unity or coherence through paraphrasing." 
One professor describes the impact of technology and how it has affected writing for his 
students. "This [skill is a] basic step for producing a comprehensive and finished piece of 
writing. It needs to be stressed now more than ever because students are composing and 
editing solely on the word-processor, which seriously degrades organization and logic. I 
require undergraduate and graduate students to provide an edited draft with their papers, so 
that they cannot avoid editing a hard-copy text." 

Faculty rated the importance of developing a chosen topic and refining goals significantly 
higher than did employers. Again, they made similar comments and illustrations as they did for 
the skill of developing paragraphs. A faculty member states "if they can't develop a topic, 
there's nothing to revise." The development of a topic is the "key writing task, especially in a 
collaborative work setting" according to a policymaker. In support of the importance of 
refining goals, faculty wrote "this is the bottom line in the writing process" and ''awareness of 
the process itself is a means to refine thinking and writing." However, an employer noted this 
skill is "less important than the content." 

Policymakers rated significantly lower than did faculty the importance of college griduates' 
abilities to consider audience and purpose to shape their voice, choose a voice to write in, and 
choose language, sentence structure and content to create that voice. 

There was also a difference between faculty and employers concerning the skill of defining 
goals as they write. Faculty rated this skill significantly more important than did employers. 
There was no significant difterence detected between faculty and policymakers. 
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In this section, the respondents agreed about the importance of eight drafting skills. They rated 
the ability to establish and maintain a focus, to quote effectively, and to avoid common 
granmiatical errors of standard written English as extremely important skills. However, they 
disagreed about six skills with faculty consistently rating these skills significantly higher in 
importance than the other respondent groups. 



A.vi. Collaborating 

The ability of college graduates to write collaboratively is advocated by a number of writing 
experts (Anderson, 1985; Barclay et al., 1991; White, 1991; and Witte, 1992). The specific 
results from Anderson's surveys indicated that writers should be able to co-author written 
material, delegate writing to others, critique others' drafts, and seek draft critiques from othert:. 
White states that these skills are particularly important in the business world. For example, 
sometimes several experts contribute particular sections to a major report based upon their 
own individual area of expertise. In another case, a superior may copyedit the written work of 
a subordinate. 

The three respondent groups agreed about the importance of one skill in this area. College 
graduates should be able to collaborate with others during reading and writing in a given 
situation (see Table 13a). In the remaining three more specific items regarding collaboration, 
there were disagreements about their importance that cut across the respondent groups (see 
Table 13b). No additional consensus was reached after the second round of surveys. 

Policymakers and employers rated significantly higher than did faculty the skill of writing 
documents for another's signature. The reality of the employers' and policymakers' 
environment requires this skill of college graduates. They note "this is what entry-level people 
often must do— and do so without embarrassment to the 'signer'," "this is done quite 
frequently in business," "as a director, I need this type of help from staff who are responsible 
for various aspects of the organizafion," "if one is interested in moving ahead [they need this 
skill]," "this is more important than 'group' writing in our organizafion," and "the most 
important documents tend to be signed by senior officers." Faculty disagree and tend to regard 
this skill as trivial and not relevant for college graduates. They remark "ghostwriting is a 
narrow specializafion; not all need to learn," "seems dishonest," "they ought to be wrifing for 
their own signature," and "sounds like a specific skill for a secretarial course." Another 
professor concludes "most writers would find this work unfulfilling. I sfiU emphasize that the 
writer must do his/her own work." 

Faculty rated significantly more important the ability of college graduates to critique other's 
drafts and the ability to negotiate critiques of v/riring from others than did the employers and 
policymakers. The faculty wrote numerous comments supporting the importance of this skill. 
They said "I believe in the power of peer revision," "[this is a] terrific strategy to help students 
become more conscious of audience, purpose, voice, and organization," "this seems difficult 
for many students to do, unfortunately. To be able to critique others' drafts, the writer must 

56 



o 66 
ERIC 



Table 13a, Analysis of Variance — Collaborating 





Mean 


Mean 

Square 

Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Collaborate with others during reading and writing in a 
given situation 


2.82 


3.26 


2.94 


.3325 


Critique others' drafts 


2.77 


19.10 


1.93 


.GOOF- 


Negotiate critiques of their writing from others 


2.86 


14.05 


2.77 


.0071* 


Write documents for someone else's signature 


4.22 


55.23 


5.37 


.0001* 


SKILLS — ROUND 2 


Critique others' drafts 


2.67 


11.20 


l.ll 


.0001* 


Negotiate critiques of their writing from others 


2.83 


9.46 


1.56 


.0029* 


Write documents for someone else's signature 


4.25 


21.00 


3.43 


.0027* 


* Significant differences noted in TUKEY and Least Square Means 



respond as writer and audience, so the learning process is taken further. Negotiating critiques 
allows the writer to see that writing is a ^product' and so possibly removes oneself from the 
personal vulnerability;' '^reading and writing are interdependent activities, and development 
occurs through peer critique of writing^ **this improves sense of audience and revision " *'if 
students cannot critique another's draft, they will scarcely be aware enough to edit their own 
work effectively. Critiquing others' work is excellent training/' and believe in the power of 
peer revision and cntical thinking skills " However, sonae faculty and members of the other 
respon Jent groups disagree. A professor remarks '^this skill] is not as important as writing on 
their own well " Other faculty believe this skill is not inaportant for everyone. In some work 
environments collaboration is less important. As an employer notes **in our organization there 
is very little 'group' writing. Employees are expected to excel on their own." 

All three respondent groups agreed that college graduates should be able to collaborate with 
others during reading and writing in a given situation. They disagreed about the importance of 
three specific collaborative skills. In two cases, faculty rated the skills significantly more 
important than did employers and policymakers. However, both employers and policymakers 
rated the ability to write documents for another's signature as more importmt than did the 
faculty. 
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Table 13b. Disagreements about Collaborating Activities 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Critique otliers' drafts 

Standard Deviation 


3.43 
1.39 


2.45 
1.44 


2.80 
1.22 


.0001 


.0198 


n.s. 


Negotiate critiques of their writing 
from others 

Standard Deviation 


3.41 
1.51 


2.57 
1.71 


2.95 
1.74 


.001 1 


n.s. 


n.s. 


Write documents for someone 
else's signature 

Standard Deviation 


3.28 
1.87 


4.84 
2.53 


3.68 
2.20 


.0001 


n.s. 


')066 


SKILLS — ROUND 2 


Critique others' drafts 

Standard Deviation 


3.16 
1.15 


2.37 
.06 


2.97 
.89 


.0002 


n.s. 


.0030 


Negotiate critiques of their writing 
from others 

Standard Deviation 


3.23 
1.27 


2.55 
1.27 


3.20 
1.16 


.0042 


n.s. 


.0109 


Write documents for someone 
else's signature 

Standard Deviation 


3.58 
1.58 


4.67 
2.08 


3.83 
1.31 


.0009 


n.s. 


.0103 


EMP = Employers: FAC = Faculty; PM = Policymakers; n.s. = not significant 



A.vii. Revising 

Researchers have discovered that high school and college writers in general do not know how 
to revise effectively (Faigley et al., 1985). Several studies have attempted to define what 
constitute these effective abilities. The more experienced writers tend to make changes that 
affect the structure and content of a text more than inexperienced writers (Beach 1976; 
Sommers, 1978, 1980). College freshmen tend to view their first drafts as conceptually 
complete and believe there is only a need for mechanical corrections. Other researchers have 
found a predominance of mechanical and word-level revisions among inexperienced writers 
(Bridwell, 1980; Perl, 1980; Pianko, 1977, 1979a, 1979b; Stallard, 1974). More experienced 
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writers described their primary objective as finding the form, shape, structure, or design of 
their argument, while the novice writers concentrated more on changing words as words, 
divorced from their role in the text. The more mature writers v/ere also more concerned about 
their audience, and their imagined reader influenced their process of revision by functioning as 
a critic. The novice writers were more concerned about following abstract learned rules about 
texts, such as standard, inflexible organizational structures for essays and paragraphs. The 
more experienced writers ''sought to discover or create meaning" through revision, while the 
novice writers ''sought to bring their v/riting into congruence with a pre-defined meaning." 
Finally, the more experienced writers often viewed revision as a process with different levels 
of attention and different agenda for each stage. For example, the more experienced writers 
often separated the content-related revision of their documents from the grammatical and 
mechanical copyediting. Studies by Flower and Hayes (1985), Schriver (1992), and Wallace 
and Hayes (1991 ) indicate the importance of global, large-scale revision to successful writing. 
However, local, small-scale revision and euiting are not to be overiooked. These are equally 
important abilities according to writing experts (Brand, 1991; Greenburg, 1988; Loackeret al., 
1984; Schriver, 1992; Wallace & Hayes, 1991; and White, 1991). 

All three respondent groups agreed that four skills were extremely important for college 
graduates to attain. They should correct grammar problems; revise to improve word choice; 
select, add, substitute, or delete information for a specified audience; and reduce awkward 
phrasing and vague language (see Table 14a). They disagreed about the importance of three 
skills including the ability to add transitions as needed, diagnose and correct problems with 
text (including matters of organization, focus, content, development, and coherence), and 
assess writing and revise it (see Table I4b). Faculty rated these three skills significantly higher 
than did employers and policymakers. No additional consensus was reached about revising 
after the second round of surveys. 

The ability to assess writing and revise it as well as the ability to diagnose and correct 
problems with the text received the strongest support from faculty. They remarked ''writing is 
revision and revision entails being a critical reader of one's own writing'' and ^'teaching 
students to be their own editors is one of my primary goals. This is what distinguishes 'fine' 
from 'average' writers." Some faculty note the difficulty of teaching students this particular 
skill. As one professor states "[this is] most crifical and a difficult step to develop, but once 
learned it informs much of everything they do." Faculty also believe that adding transitions is 
extren^.ely important. They note "transitions are essential to flow, logical organization, and 
reader-based wridng ' and that *4deally the transition should be in the original drafts." 
However, employers and policymakers rated these skills as less important. They believe that 
not all graduates need revision skills. Furthermore, in many company and policy units, there is 
not an opportunity or the fime to revise written documents. In this secfion, the ability to correct 
grammar problems was rated the most important followed by the ability to reduce awkward 
phrasing and vague language. Furthermore, all of three groups agreed that revising to improve 
word choice and selecting, adding, substitutions, or deleting information for a specified 
audience are crifical skills. 



59 

ERLC 



Table 14a. Analysis of Variance — Revising 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILL— ROUND I 




Correct grammar problems 


1.55 


.13 


,66 


.8223 


Revise to improve word choice 


1.92 


1.66 


I.IO 


.2232 


Select, add. substitute, or delete information for a specified 
audience 


2.30 


1.05 


1,54 


.5075 


Reduce awkward phrasing and vague language 


1.89 


1.22 


.88 


.2508 


Assess their own writing and revise it 


1.64 


14.42 


.71 


.0001* 


Diagnose and correct problems with the text, including 
matters of organization, focus, content, development, 
and coherence 


1.73 


8.41 


.74 


.0001* 


Add transitions between piirts of the text as needed 


2.19 


21.51 


1.35 


.0001* 


SKILLS — ROUND 2 


Assess their own writing and revise it 


1.81 


7.28 


.58 


.0001* 


Diagnose and correct problems with the text, including 
matters of organization, focus, content, development, 
and coherence 


1.84 


6.64 


.63 


.0001* 


Add transitions between parts of the text as needed 


■ 04 


5.42 


.74 


.0009* 


* Significant differences noted in TUKEY and Least Square Meai 



A.viii. Features of Written Products 

Many studies, especially those that involve surveys and large-scale assessments of written 
texts themselves, indicate specific, detailed features of texts that illustrate writing ability. 
Cullen et al. (1987) found the critical skills to be the "main point is clearly stated or implied," 
"essay is grammatically error-free," "essay demonstrates effective use of sentence variety," and 
"essay demonstrates precise and sophisticated word choice, appropriate to the level of style." 
Clarity, conciseness, organization, and grammar were cited as important skills by respondents 
in .several surveys (Barman & Fischer, 1984; Bennett & Olney, 1986; Stine & Skarzenski, 
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Table 14b, Disagreements about Revising 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS -ROUND 1 


Assess their own writing and revise il 
Standard Deviation 


2.12 
.99 


1.33 
.66 


1.93 
1.07 


.0001 


n.s. 


.0016 


Diagnose and correct problems with 
the text, including matters of 
organization, focus, content, 
development, and coherence 

Standard Deviation 


2.10 
1.00 


1.49 
.75 


1.93 
.94 


.0001 


n.s. 


.0100 


Add transitions between parts of the 
text as needed 

Standard Deviation 


2.66 
L26 


1.80 
L05 


2.73 
1.34 


.0001 


n.s. 


.0002 


SKILLS -ROUND 2 


Assess their own writing and revise it 
Standard Deviation 


2.16 
1.07 


1.56 
.56 


2.13 
.82 


.0010 


n.s. 


.0010 


Diagnose and correct problems 
with the text, including matters of 
organization, focus, content, 
development, and coherence 

Standard Deviation 


2.07 
.96 


1.60 
.53 


2.27 
1.17 


.0041 


n.s. 


.0050 


Add transitions between parts of 
the text as needed 

Standard Deviation 


2.28 
1.12 


1.82 
.69 


2.40 
.93 


.0169 


n.s. 


.0033 


EMP = Employers: FAC = Faculty; PM = Policymakers; n.s. = not significant 



1979; and Storms, 1983). Business executives in Iowa cited the most frequent writing 
problems as wordiness, grammar, sentence structure, spelling, clarity, and organization (Stine 
& Skarzenski, 1979). Other studies (Haswell, 1984; White & Polin, 1986: and Witte et al., 
1982) continue to add important dimensions of good written products that were incorporated 
into our survey. 

Faculty, employers, and policymakers agreed that seven skills are important in this section of 
the survey (see Table 15a). The ability to use correct grammar, syr tax, (word order), 
punctuation, and spelling was rated extremely important followed by the use of language that 
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Table 15a. Analysis of Variance — Features of Written Products 







Mean 
Model 


Mean 
Sc|UtU*e 
Error 


PR > F 


SKILL— ROUND 1 










Use active or passive voice where appropriate 


2.65 


3.85 


1 ?2 


.1792 


Use language their audience understands 


1 .73 






. /Uo4 


Define or explain technical terms 


2.0O 




I At 




Use concise language 


1.83 


.17 


1.04 


.8502 


Use correct grammar, syntax (word order), punctuation, 
and spelling 


1 .50 


.41 


"7*7 


J 1 1 


Use correct reference forms 


2.41 


1.29 


2.06 


.3.') Do 


Use the specific language conventions of their academic 
discipline or professional area 


111 


4.91 


2.56 


.1496 


Vary sentence length and style for rhetorical purposes 


2.88 


28.05 


2.49 


.0001* 


Use visual aids, tables, and graphs 


3.74 


24.23 


3.60 


.0015* 


SKILLS — ROUND 2 


Vary sentence length and style for rhetorical purposes 


2.85 


10.05 


0.96 


.0001* 


Use visual aids, tables, and graphs 


3.74 


15.58 


2.25 


.0013* 


* Significant differences noted in TUKEY and Least Square 


Means 









the audience understands, and use of concise language. The remaining three skills were rated 
with medium importance and included the ability to use active or passive voice where 
appropriate, define or explain technical terms, use correct reference forms, and use the specific 
language conventions of the academic discipline or professional arena. Faculty, employers, 
and policymakers continued to disagree in the second survey about the importance of the 
ability to vai7 sentence length and style, and the skill of u.sing visual aids, tables, and graphs 
(.see Table 15b). 

Employers rated significantly higher than did faculty the need for college graduates to use 
supporting visual aids in their written products. They .stress that this adds clarity to 
presentations and is critical for employees in the workplace. The supporting comments from 
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Table 15b. Disagreements about Features of Written Products 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PVI 


FAC/PM 


SKILLS — ROUND 1 


Vary sentence length and style for 
rhetorical purposes 


.164 


2.46 


3.05 


.0001 


n.s. 


.0252 


Standard Deviation 


1.76 


1.54 


1.38 








Use visual aids, tables, and graphs 


3.17 


4.15 


3,28 


.0008 


n.s. 


.0103 


Standard Deviation 


1,67 


2.03 


i,77 








SKILLS — ROUND 2 


Vary sentence length and style tor 
rhetorical purposes 


3.19 


2.56 


3.33 


,0027 


n.s. 


.0001 


Standard Deviation 


1,16 


.93 


^84 








Use visual aids, tables, and graphs 


3.07 


4,07 


3.57 


,0005 


n.s. 


n.s. 


Standard Deviation 


1,47 


1.58 


1-22 








EMP = Employers; FAC = Faculty; PM = Policymakers; n.s. = not significant 



employees describe the importance of this skill as follows: **backbone of the presentation''; 
'^extremely important in the business world — sometimes the difference between good and 
great''; **modem communications require the use of these tools''; and '^critical because this also 
implies that you know what your audience likes and you use it." Employers emphasized that 
this skill is gaining in importance especially because employees now use microcomputers in 
much of their daily work. They expect this skill to become increasingly important in the 
future. However, faculty disagree about the importance of this skill. Many faculty believe this 
skill is only important for business or technical writing courses. Other faculty expect that 
employers will train new hires to acquire this skill. They believe the use of visual aids is 
heavily dependent upon the particular context or discipline and that some students will never 
need this skill. The faculty tended to mte this skill lower because they did not view it as 
essential for all college graduates nor for all writing situations or tasks. 

Faculty rated the ability to vary sentence length and style for rhetorical purposes as 
significantly more important than did both employers and policymakers. Faculty viewed this 
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skill as a mandatory requirement while the other respondent groups thought it was not 
essential. The faculty connmented "this is one sign of sophisticated writing"; "variety 
demonstrates understanding and adds to interest**; and "fluidity occurs through learning 
different stylistic connections.'' 

A.ix. Written Products 

A study of writing skills at the college level must include not only how students write, but also 
what they should be able to write after completing their education. Faigley's 1981 survey of 
200 college-graduate employees indicated that "the ability to use specific business and 
technical writing document forms'' was one of the most important skills that should be taught 
in college writing courses. Storms (1983) surveyed 804 graduates of the college of business 
administration at Miami University of Ohio, who reported that they write memoranda, letters, 
short reports, step-by-step instructions or procedures, and proposals. In another study, senior 
executives reported that the most frequently used forms of business communication among 
some of the largest United States' corporations were memoranda, letters, analytical reports, 
and informational reports (Bennett and Olney, 1986). Anderson (1985), Faigley et al. (1981), 
and Storms (1983) found that the majority of college graduates in the workforce indicated that 
they write as many as seven or eight different types of letters, memoranda, and reports. These 
various types of written products were incorporated in our Delphi survey. 

In this section of the survey, faculty, employers, and policymakers agreed that nine skills were 
important for college graduates to attain with a minimum amount of training in the workplace 
context. These included the ability to write memorandum, letters, formal reports, summaries of 
meetings, scripts for speeches/presentations, and complete pre-printed forms that require 
written responses. After the second round of surveys, they agreed that three additional skills 
were important (see Table 16a). College graduates should be able to write step-by-step 
instructions, journal articles, and policy statements. However, they disagreed about the 
importance of college graduates writing abstracts and evaluations (see Table 16b). 

Faculty rated significantly higher than did employers and policymakers the need for college 
graduates to write abstracts. Faculty comment that this skill is ' a very important skill, because 
the genre is very important — nobody reads the whole report," "a writer of good abstracts has 
learned to read effectively," "college graduates should be able to see the essential argument in 
someone's writing and sort it out from the less important," "summarizing and condensing are 
often necessary to cope with the information overload," and "writing abstracts trains writers to 
identify main points in their own writing and others' writing." Employers and policymakers 
believe this skill is too advanced for college graduates and that only supervisors or individuals 
in advanced managerial levels of the organization need this skill. Many believe this skill is not 
necessary for the entry level new employee. 

Policymakers rated the ability to write evaluations as significantly less important than did 
faculty. "These skills are appropriate for anyone who wants to move up vocationally," and it 
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Table 16a* Analysis of Variance — Written Products 







Mean 
Square 
Model 


Mean 

Square 

M.^an 


Pr>F 
Error 


SKILL— ROUND 1 


Write memoranda 


1.81 


1.96 


1.46 


.2627 


Write letters 


1.66 


.01 


.98 


.9918 


Write formal reports 


2.37 


3.51 


1.95 


.1676 


Write summaries of meetings 


2.42 


1.87 


2.51 


.4749 


Write scripts for speeches/presentations 


3.56 


4.19 


3.66 


.3202 


Complete pre-printed forms that require written responses 


2.39 


.10 


3.19 


.9678 


Write step-by-step instructions 


2.29 


5.34 


1.96 


.0670* 


Write journal articles 


4.01 


16.50 


4.31 


.0232* 


Write abstracts 


3.34 


67.32 


3.24 


.0001* 


Write policy statements 


3.41 


28.35 


3.79 


.0007* 


Write evaluations 


2.73 


10.03 


2.39 


.0161* 


SKILL— ROUiND 2 


Write step-by-step instructions 


2.31 


2.24 


.90 


.0858 


Write journal articles 


4.41 


1.74 


2.14 


.4437 


Write policy statements 


3.98 


3.48 


1.70 


.1316 


Write abstracts 


3.78 


9.01 


1.58 


.0040* 


Write evaluations 


3.02 


3.74 


1.34 


.0637* 


* Significant differences noted in TUKEY and Least Square Means 



"teaches clarification of criteria and ways those have been met'' state faculty. Another 
professor remarks ''College graduates from this four-year liberal arts institution should be able 
to have the writing skills necessary to perform these tasks at a high level of competence. That's 
why my grading is above the mean in all cases. Students are asked to do all these things in 
their classes." However, the other two respondent groups again stressed that the ability to write 
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Table 16b. Disagreements about Written Products Between Respondent Groups 





N4eans 


Significance Level 


EMP 


FAC 


PM 




E/P 


F/P 


SKILLS — ROUND 1 


Write step-by-step instructions 

Standard Deviation 


2.66 
1.62 


2.15 
1.32 


2.18 
1.25 


.0410 


n.s. 


n.s. 


Write journal articles 

Standard Deviation 


4.66 
2.07 


3.76 
2.12 


3.83 
1.93 


.0084 


.0456 


n.s. 


Write abstracts 

Standard Deviation 


4.60 
2.13 


2.75 
1.66 


3.28 
1.69 


.0001 


.0009 


n.s. 


Write policy statements 

Standard Deviation 


4.24 
2.15 


3.05 
1.90 


3.28 
1.75 


.0005 


.0166 


n.s. 


Write evaluations 

Standard Deviation 


3.16 
1.72 


2.47 
1.46 


2.90 
1.52 


.0101 


n.s. 


n.s. 


SKILLS — ROUND 2 


Write abstracts 

Standard Deviation 


4.21 
1.24 


3.50 
1.35 


4.07 
.91 


.0032 


n.s. 


.0103 


Write evaluations 

Standard Deviation 


3.16 
1.45 


2.85 
1.06 


3.38 
.98 


n.s. 


n.s. 


.0154 


EMP = Employers; FAC = Faculty; PM = Policymakers; n.s. = not significant 



evaluations is most needed by upper level supervisors with more work experience than the 
recent college graduate has acquired. 

A.x. Factor Analysis 

The items included in the writing survey were further analyzed by conducting a principle- 
components factor analysis with varimax rotation. This procedure was used to test the validity 
of the proposed structure of the underlying variables and to gain further insights into these 
constructs. For the writing area, nine factors were extracted since the original 
conceptualization consisted of nine dimensions that were believed to account for the writing 
variables. The rotated factor matrix is illustrated in Tables 17a through 17d, 
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Table 17a. Writing Goals Survey — Factor Analysis — Factors 1 and 2 



Goals 


Factor 1 


Goals 


Factor 2 


Include clear statement of main ideas 


.77369 


Draw on their individual creativity and 
imagination 


.69143 


Select, organize, and present details to 
support a main idea 


.70121 


Create ideas for their writing 


.67435 


Organize their writing in order to 
emphasize the most important ideas 


.63201 


Make appropriate use of creative 
techniques of humor and eloquence 


.65321 


Link key points in an argument with 
multiple points 


.62906 


Analyze their own experience to provide 
ideas for their writing 


.63716 


Cluster similar ideas 


.58723 


Use an appropriate tone of voice 


.63694 


Demonstrate patterns of reasoning 


.55455 


Use appropriate vocabulary 


.59343 


State their purpose(s) to their 
audience 


.54073 


Arrange words within sentences to fit the 
intended purpose and audience 


.56825 


Recognize the rhetorical problem and 
focus and narrow their plan 


.52826 


Consider audience and purpose to choose 
voice, language, and structure 


.56206 


Refine goals as they write 


.52221 


Retrieve material from their memory 


.53078 


Maintain coherence among sentences, 
paragraphs, and sections 


.51872 


Use knowledge of audience expectations 
and values to shape a text 


.49830 


Identify problems to be solved that their 
topic suggests 


.49465 


Select, add, substitute or delete 
information for a specific audience 


.47923 


Develop patterns of organization for 
their ideas 


.48436 


Vary sentence length and style for 
rhetorical purposes 


.46664 


Establish and maintain a focus 


.47133 


Be aware of multiple purposes and goals 


.46564 


Move between more abstract and more 
specific levels of argument 


.45655 


Plan writing processes with effective 
strategies and techniques 


.45196 


Develop their chosen topic 


.45591 


Write effective introductions and 
conclusions 


.44725 


Provide a context for the document in the 
introduction 


.45275 


Refine the notion of audience(s) as they 
write 


.41162 


Locate and present adequate supporting 
material 


.42763 






Assess and revise their own writing 


.37307 






Percent of variance accounted for by factor 


8.25 


Percent of variance accounted for by factor 


7.54 
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Table 17b* Writing Goals Survey — Factor Analysis — Factors 3 and 4 



Goals 


Factor 3 


Goals 


Factor 4 


Use correct grammar, syntax, 
punctuation, and spelling 


.78088 


Write policy statements 


.78836 


Correct grammar problems 


.75110 


Write evaluations 


.73155 


Avoid common grammatical errors 


.71572 


Write summaries of meetings 


.68555 


Revise to improve word choice 


.64227 


Write scripts for speeches and 




IXV^VlU^C^ Cl rV Fv VV cH VJ |JlHclotll^ cllIU Vcl^UC 

language 


,60126 


Writf* formiil rpnnrtv: 




\r'ff^intJ< Jn fnh.^^rpnfp u/tthin 'cpntpnppvi 


.57036 


Wrttp iniirnnl nrtirlp'c 




Use concise language 


.50098 


Write abstracts 


.66190 


Diagnose and correct problems 


.4861 1 


Write step-by-step instructions 


.63449 


Add transitions to text as needed 


.46365 


Write memoranda 


.50425 


Use active or passive voice where 
appropriate 


.45266 


Write letters 


.43895 


Define or explain technical terms 


.41281 


Use knowledge of their subject matter to 
shape the text 


.30307 


Use language the audience understands 


.38250 






Complete pre- printed forms that require 
written responses 


.37554 






Use correct reference forms 


.35329 






Write effectively under pressure tUid 
meet deadlines 


.31494 






Percent of variance accounted for by factor 


6.99 


Percent of variance accounted for by factor 


6.84 



Many of the variables from the ^'organizing'' section loaded most heavily on Factor I These 
included the ability to include clear statements of the main ideas; select, organize and present 
details to support a main idea; organize writing to emphasize the most important ideas; 
maintain connections that link key points in an argument; demonstrate patterns of reasoning; 
and cluster similar ideas. In addition, three variables originally in the '*pre-writing" section 
loaded onto this factor as well as three skills from the ^'drafting" section. These skills included 
locating and presenting adequate material, focusing and narrowing a plan by recognizing the 
rhetorical problem to be solved, identifying problems to be solved that the topic suggests, 
developing their chosen topic, refining goals as they write, and establishing and maintaining a 

68 



78 



Table 17c* Writing Goals Survey — Factor Analysis — Factors 5 and 6 



Goals 


Factor 5 


Goals 


Factor 6 


Understand their audiences' values, 
attitudes, goals, and needs 


.70229 


Write informative headings that match 
their audiences' questions 


.76547 


Define their anticipated multiple 
audiences 


.69968 


Demonstrate their organization by using 
informative headings 


.73768 


Address audiences whose backgrounds 
in the topic vary 


.68069 


Set up sign posts such as table of 
contents, indexes, and side tabs 


.73237 


Consider how an audience will use the 
document 


.64919 


Use visual aids, tables, and graphs 


.72355 


Address audiences whose cultural and 
communicating norms differ 


.61366 


Write documents for someone else's 
signature 


.56633 


Choose words that their audience can 
understand 


.56916 


Create and use an organizational plan 


.48543 


Understand the relationship between the 
audience and themselves 


.54602 


Organize material for more than one 
audience 


.47482 


Understand the relationship between the 
audience and subject material 


.51071 


Follow the language conventions of their 
discipline or professional area 


.35171 


Percent of variance accounted for by factor 


5.73 


Percent of variance accounted for by factor 


5.00 



focus when writing. The variables in this factor accounted for approximately eight percent of 
the variance. 

Factors 2 through 5 each accounted for roughly seven percent of the variance. The writing 
variables that loaded most highly on Factor 2 were primarily items in the "purpose for writing'' 
section and the "pre-writing" activities. Factor 3 v/as composed mainly of writing variables 
from the **revising'' and '^features of the written product'' sections of the survey. Factor 4 
consisted of solely "written product" variables. The majority of "awareness and knowledge of 
audience'' variables loaded onto Factor 5. The remaining four factors each accounted four 
percent or less of the variance. Variables in the '^organizing" section which referred to 
audience loaded onto Factor 6 while four ^'drafting" variables loaded onto Factor 7. The 
majority of the ^'collaborating" variables loaded onto Factor 8. 

This factor analysis illustrates that many of the variables loaded onto a factor that they were 
originally grouped with in the survey. However, in some sections certain features overiapped 
such as the audience-related topics that appeared in two different sections and in two different 
factors. The consideration of purpose when writing could be incorporated with prewriting 
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Table lid. Writing Goals Survey — Factor Analysis — Factors 7, 8, and 9 



Goals 


Factor 7 


Goals 


Factor 9 


Read and revise drafts to emphasize 
key points 


.68434 


Research their subject 


.44271 


Make general and specific revisions 
while v/riling their drafts 


.67851 


Quote accurately 


.41722 


Write drafts nf thpir work 


.66488 






Develop paragraphs 


.63550 






Clarify their policy and position before 
writing 


-.43857 






Percent of variance accounted for by factor 


3.89 


Percent of variance accounted for by factor 


2.20 


Goals 


Factor 8 






Negotiate critiques of their writing 
from others 


.73022 






Critique other's drafts 


.70490 






Collaborate with others 


.64995 






Discuss their piece of writing with 
someone to clarify ideas 


.55942 






Percent of variance accounted for by factor 


3.89 







activities. Writing is not a set of discrete, unrelated processes. However, for the purposes of 
developing this survey, categories were useful and provided clarity as the respondents 
completed the instrument. They stressed that if no headings or categories were used in the 
survey, they would probably not complete it. Nine different factors or groupings may not be 
necessary given the results from this analysis. 



A.xi. Reliability 

The results of the reliability analysis for the writing survey are presented in Table 18. The 
"pre-writing'' section had the highest reliability {a= .90) while the ^'collaborating ' section had 
the lowest reliability {a= JO). Overall, the reliability of the individual sections in the writing 
survey tended to increase with the number of items comprising a given section. The majority 
of reliability coefficients were above a- .80. For the purposes of most studies, reliability 
coefficients greater than a = .65 were deemed satisfactory (Mehrens & Lehmann, 1973). 



70 



SO 



Table 18. Reliability of Items in Writing Survey 



Sections 


Number of Items 


Alpha 


Awareness aiid knowledge of audience 


8 


.8657 


Purpose for writing 


7 


.8039 


Pre- writing activities 


10 


.8041 


Organizing 


18 


.8999 


Drafting 


13 


.8695 


Collaborating 


4 


.7020 


Revising 


7 


.8733 


Features of written products 


9 


.8296 


Types of written products 


11 


.8876 


Total writing survey 


87 


.9650 



A.xii. Summary 

The findings from this writing survey illuminate the most critical skills which faculty 
employers, and policymakers believe college graduates should possess. The purpose of this 
study was to find the areas of consensus shared by all three stakeholder groups. These key 
areas represent specific definitions of the elements considered important for effective writing. 

In each section of the survey, there were areas where these three groups did reach a consensus 
about the relative importance of certain skills. Faculty, employers, and policymakers agree that 
audience awareness is an important skill especially in terms of specific abilities that include 
considering how an audience will use the document, choosing words that their audience will 
understand, and understanding the relationship between audience, subject material, and 
themselves. Furthermore, considerations of audiences' values, attitudes, goals, needs, cultural 
and communication norms are important. These results are consistent with the previous 
frameworks of writing that stress the importance of college graduates' abilities to develop a 
representation of the potential readers of a text (Faigley, 1985; Flower & Hayes, 1980b; Odell, 
1981, Odell & Goswami 1985; Sommers, 1980). However, the importance of students' 
abilities to define multiple audiences and to address audiences whose backgrounds in topics 
vary widely is contested among the respondents. Faculty tend to rate these two skills as 
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extremely important academic objectives while many employers and policymakers disagree. 
Part of this lack of agreement is based upon the belief held by employers, policymakers, and a 
minority of faculty that these skills are extremely difficult for new graduates to acquire without 
substantial experience and development within their professional positions in the workplace. 
Other employers noted that their workplace was highly specialized so that there were not 
diverse audiences to interact within those situations. Here the context of the individual 
workplace of the employer did affect the importance of these two particular skills. Employers 
with more diverse clients or customers tended to rate these skills as very imponant. 

Some respondents were critical that these two skills are not taught in the classroom. Students 
tend to write for one audience which is mainly their instructor who reads and evaluates their 
work. Even several faculty noted the challenges associated with teaching students to write for 
real audiences with varying levels of background in the topic area. 

Writers do need to clearly state their purpose to their audiences in a manner that is consistent 
with their chosen topic according to the participants in this study. Again, this is consistent with 
the previous theoretical models (e.g., Faigley et al., 1985; Flower & Hayes, 1980a) and the 
outcomes from their research. The ability to use creative techniques of humor and eloquence 
are considered to be of medium importance. 

A similar theme of disagreement emerges in the skills associated with defining the purpose of 
writing that was also evident in the awareness of knowledge area. Should college graduates be 
aware of multiple purposes and goals? The notion of multiple purposes troubles some faculty, 
employers, and policymakers. They believe that multiple purposes is an extreme standard that 
is difficult for students to achieve in their collegiate education. They would be satisfied if 
students could clearly state one purpose. College graduates achieve this skill through 
experience in the workplace according to some participants. However, this was a skill that 
faculty rated significantly higher than did the other respondent groups. 

The pre-writing phase involves many important skills. College graduates should te able to 
research their subject, identify problems to be solved that their topic suggests, aad discuss 
their writing with someone to clarify what they wish to say. This planning phase helps writers 
to prepare for their writing by thinking about a process and design to express their own ideas 
(Camevale et al., 1990; Flower & Hayes, 1980b; Loacker et al., 1984). The respondents 
disagreed about the importance of nearly three-quarters of the specific pre-writing skills (see 
Table 19). They disagreed about the importance of analyzing experiences to provide ideas for 
writing as well as creating ideas or retrieving material from memory. Some employers note 
that their workers do not need to create ideas or analyze their own experiences. Their 
responses were influenced by the organizational contexts within which they hire new 
employees. The faculty highly value these creative abilities. 

A controversial skill in writing seems to be whether college graduates need to clarify a policy 
and position before writing. Respondents in all three groups disagreed about when an 
individual needs to clarify his or her position. Many individuals believe this ability is 
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Table 19. Summary of Consensus and Disagreements in Each Section of Writing Survey 



Section oT Survey 


Round 1 — 
Number of Items 


Round 2 — 
Number of Items 


Final — 
Number of Items 


A 




D 




A 


% 


D 




A 




D 


% 


rVWdlCilCSh dilU K.nOWiCUi^C 

of audience 


3 


37.5 


5 


62.5 


3 


60.0 


0 


40 


6 


75.0 


2 


25.0 


Purpose for writing 


4 


57.1 


3 


42.9 


1 


33.3 


2 


66.7 


5 


71.5 


2 


28.5 


Prewriting Activities 


3 


30.0 


7 


70.0 


0 


0.00 


7 


100,0 


3 


30.0 


7 


70.0 


Organizing 


6 


33.3 


12 


66.7 


8 


66.7 


4 


33,3 


14 


77.8 


4 


22.2 


Drafting 


5 


38.5 


8 


61.5 


3 


37.5 


5 


62.5 


8 


61.5 


5 


38.5 


Collaborating 


I 


25,0 


3 


75.0 


0 


0.0 


3 


100.0 


1 


25.0 


3 


75.0 


Revising 


4 


57.1 


3 


42.9 


0 


0.0 


3 


100.0 


4 


57.1 


3 


42.9 


Features of Written Products 


7 


77,8 


0 


22 2 


0 


0.0 


2 


1(X).0 


7 


77.8 


0 


22.2 


Written Products 


6 


54.6 


5 


45.4 


3 


60.0 


2 


40.0 


9 


81.8 


2 


18.2 


70TAL 


39 


44.8 


48 


55.2 


18 


37.5 


30 


62.5 


57 


65.5 


30 


34.5 


A = Number of Items for which there v/as agreement; D = Number of Items for which there was disagreement 



important throughout the writing process while others think the ability to clarify is very 
important prior to writing. 

The ability to clearly organize and structure a document is essential (Faigley et al., 1981; 
White & Polin, 1986; Wine et al., 1982). College graduates should be able to maintain 
coherence within and among sentences, paragraphs, and sections of writing as well as include 
clear statements about main ideas. The ability to write informative headings to match 
audience's questions were rated of medium importance. Faculty rate significantly higher than 
did employers and policymakers the writing goals of linking key points in an argument; 
moving between abstract and specific levels of argument; selecting, organizing and presenting 
details to support a main idea; and using knowledge of subject matter to shape a text. 

Many students complete an initial draft of their written work and in the classroom have 
opportunities to make revisions. Flower and Hayes (1980b) as well as Faigley et al. (1985) 
note the differences between skilled writers and less experienced writers. The more skilled 
writers are able to bette. generate ideas and make revisions based upon the relationship 
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between the reader and the writer. Less-skilled writers are more likely to recall simple facts as 
they develop topics. The respondent groups believe that certain fundamental or basic skills are 
extremely important such as quoting accurately, avoiding common grammatical errors of 
standard written English, establishing and maintaining a focus, writing effective introductions 
and conclusions, and writing effectively under pressure to meet deadlines. 

The main area of contention among the respondent groups centered around the frequency and 
time involved with revising a written product. While the faculty rate this skill as very 
important, some employers and policymakers disagree. Some supervisors did not believe that 
their organization fostered attention or time to the development of multiple drafts of 
documents that would facilitate revisions. Some individuals seem to expect employees to write 
good documents upon the initial attempt while faculty encourage multiple cycles of revisions. 

Collaboration with others during reading and writing is another goal valued by many faculty. 
However, some employers and policymakers, due to their workplace environments, 
emphasized that collaboration was not expected or realistic due to the nature of an individual's 
job responsibilities. In some organizations, individuals are expected to work on their own 
rather than in teams. Employers and policymakers did rate the importance of writing 
documents for another's signature significantly higher than faculty did. Again, this difference 
can be attributed, at least in part, to the differences in organizational contexts. Employers and 
policyraal^ers stress that this skill in reality is important for success in some companies. 

There are certain features of the written products that all respondent groups believe are 
important. College graduates should be able to use active or passive voice where appropriate, 
use correct granmiar, use specific language conventions of their academic discipline, and use 
language that their audience understands. However, employers rate the importance of using 
visual aids, tables, and graphs as significantly more important than do faculty. There are also 
actual written products that the participants rate as important documents that college graduates 
should be able to develop with minimal training in the workplace. These products include 
memorandum, letters, formal reports, summaries of meetings, scripts for speeches, and the 
completion of pre-printed forms. However, faculty rate significantly higher than did employers 
and policymakers the ability to write abstracts and evaluations. 



A.xii.b. Advanced Writing Skills 

In every area of writing, there are certain skills that all three respondent groups believe are 
very important for college graduates to achieve in order to be effective employees and citizens. 
These skiUs range ft-om very basic and fundamental to more advanced levels. The basic skills, such 
as quoting accurately and avoiding common grammatical errors, or using concise language, are 
extremely important according to the stakeholders in higher education. These basic skills were 
included in this sttidy since they were cited in the literattire. While faculty may consider these skills 
as too elementary for college students, some employers note that their new employees arc lacking 
adequate preparation in some of these areas that hinders their effectiveness in the workplace. 
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The particular challenge to higher education is how we can better prepare our students to 
acquire advanced writing skills. The ability to write clearly is more than simply conveying 
information. College graduates with advanced skills analyze their readers' needs, values, 
attitudes, goals, and expectations as they create their text. Based on this analysis, college 
graduates make reasoned judgments about how to structure, organize, and develop their ideas 
in relation to their audience, themselves, and their subject materials as well as their purpose for 
writing. College graduates understand the relationship between the purpose of their 
^nmmunication and the problems or issues that need to be resolved in achieving that purpose. 

College graduates with advanced writing abilities also become their own critics. They evaluate 
their own writing and recognize confusing or vague language that requires clarification to 
increase the reader's comprehension. They also learn to cluster similar ideas and categorize 
information. When college graduates review their own work, they know what the readers may 
not understand and they define or explain technical terms. They can find examples to remove 
ambiguities. In general, college graduates should learn to assess their own writing and find 
ways to correct problems that may exist. When college graduates are their own critics, they 
willingly correct problems and learn from their mistakes. 

When college graduates are developing their own positions, they can use writing as a method 
to claiify their views. They locate and present material to support or justify their major points. 
They select evidence or further information that supports their main ideas. They draw 
conclusions that are supported by relevant reasons. In order to draw these conclusions, college 
graduates must evaluate the credibility, accuracy, and reliability of the various sources of 
information that they review. They analyze and evaluate their own arguments or positions to 
confirm or dis-confirm their own reasoning. College graduates also draw upon their creativity 
and imagination to engage the audience. They are able to analyze their own experiences to 
provide ideas for their writing. 

College graduates with advanced writing abilities possess dispositions associated with critical 
thinking. They are open-minded and strive to understand and consider divergent points of 
view. This disposition is essenfial, if college graduates are to be able to evaluate norms 
different from their own to reshape or write a document for different audiences who have 
different cultural and communication norms than the writer. The tendency to be fair-minded in 
seeking the truth and impartial is considered important especially when the findings of an 
inquiry may not support a writer's preconceived ideas. Flexibility and adaptiveness to change 
are two additional tendencies that help writers to find ways to cope effectively with different 
audiences' needs and expectations. Equally important, college graduates should find ways to 
collaborate with others as they are writing and developing their ideas. 

When college graduates are drafting and revising their writing, they are aware of and act on 
the interrelationships between the readers, the subject matter (text), and themselves rather than 
focusing on one area in isolation. They consider the audience and purpose to shape their voice 
when writing. 
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The ability to write effectively and in an advanced manner is intricately tied v/ith the writer's 
critical thinking abilities. College graduates must move beyond conveying or restating other s 
facts and ideas. Effective writers learn how to analyze and evaluate information while they 
concentrate on the interrelationships between themselves, the readers, the text, and their 
purposes for writing. 

Writing is a complex process and has been studied using a variety of methodologies. The 
survey approach used in this project imposes some limitations on the depth of knowledge that 
can be gained about these skills. However, this survey process involved the participation of a 
large group of diverse individuals representing a variety of organizations. These results reflect 
the perspectives of this group of participants. We make no claim that the writing skills agreed 
upon in this study will continue to have a consensus among an even larger group of 
stakeholders. These results provide information that others can consider and critique in order 
to determine its applications within their own settings. 

The outcomes from this survey do not suggest that all institutions should have the same 
curricular goals and expected outcomes for their college graduates. Instead, this work points 
the way to essential skills that faculty may wish to consider when they arc making revisions to 
the curriculum or their own classes. It also begins a broader review and dialogue about critical skills 
among diverse groups of individuals who are comm^itted to improving undergraduate education. 

The writing process itself as taught at the collegiate level has the potential to develop 
undergraduates since it requires them to think, question, and analyze. As college students write 
their ideas into arguments or positions, they are at the same time clarifying their values and 
developing their character. As Marshall Gregory (1994, 34) notes "it is not too much to expect 
that students who work hard at learning to write will improve their powers of reasoning and 
judgment, refine their recognition of good reasons and shapely argument, and strengthen their 
respect for well-used language and verbal discourse." The ultimate goal is to improve student 
performance in writing so that college graduates become more effective communicators at 
work and in society. 

B. Speech Communication Skills 

B.i. B asic Speech Communication Skills 

The first section of the inventory focuses on basic speech communication skills. The sub-skills 
in this part of the survey relate to selecting and anranging elements to produce spoken 
messages. This section is further subdivided into a general section, followed by message 
development and organization, content and situation analysis, message support, and finally, 
message type. 
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B.i.a. Basic Speech Communication Skills — General 



The respondents to the initial survey agreed about the level of imponance for four skill 
statements in this section (see Table 20a). The skill ranked with greatest importance, "college 
graduates should be able to state ideas clearly" had a substantial amount of support in the 
literature (see, for example, Bassett, Whittington, & Stanton-Spicer, 1978; Bienvenu, 1971; 
Ewens, 1979; Morreale, 1990; and Rubin, 1982), Respondents ranked coniniuni ating 
ethically as the second most important skill, followed by recognizing when it is inappropriate 
to communicate and communicating candidly (in an open and direct manner). 

The two contended statements in this subsection both relate to the issue of communication 
ethics (see Table 20b). Faculty and policymakers disagreed about the degree to which 
graduates should be expected to accept responsibility for their own communication behavior. 
Faculty rated this skill significantly higher than did policymakers. The second statement 
concerning awareness of language indicating bias on gender, age, ethnic, or sexual/affectional 
orientation was drawn from both Ruben (1976) and Hymes (1986). In this case, employers 
rated the skill significantly higher than did policymakers. By the second survey round, all three 
groups agreed that accepting responsibility for one's own communication behavior and being 
aware of language indicating bias were extremely important (see Table 20a). 

B.i.b. Basic Speech Communication Skills - Message Development and Organization 

The ability to choose appropriate and effective organizing methods is a critical skill (Aitken & 
Neer, 1992; Backlund, Brown, Gurry & Jandt, 1982; Bassett et al„ 1978; Di Salvo & Backus, 
1981; Johnson & Szczupakiewicz, 1987; Muchmore 8l Galvin, 1983; Murphy & Jenks. 1982; 
Rubin, 1982; Witkin, 1973). This involves the identification and attainment of communication 
goals (Aitken & Neer, 1992; Boileau, 1982; Camevale, Gainer, & Meltzer, 1990; and 
Wiemann 1977a, 1977b). 

Faculty, employers, and policymakers agreed obout the importance of four skills after the first 
round of sui*veys. The most important skills were the abilities to structure a message for 
effectiveness and choose appropriate organizing methods (see Table ? la). Other skills rated as 
important were using summary statements and outlining key points. 

Among the contended statements of this subsection, a disagreement arose from the importance 
of choosing the topic and message about which graduates ai*e comfortable and knowledgeable 
to speak (see Table 21a). There were statistically significant differences among all three of the 
re^ipondent groups with faculty ranking it as most important followed by policymakers then 
employers. This was the only skill in the message development and organization section for 
which there were significant differences between employers and policymakers. 
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Table 20a. Analysis of Variance in Basic Speech Communication Skills — General 





Mean 


Mean 
Square 
Model 


Mean 
Error 
Square 


Pr>F 


SKILLS — ROUND 1 


Recognize when it is appropriate to communicate 


2.20 


2.19 


1.4-; 


.2213 


Communicate ethically 


1.54 


1.65 


.63 


.0768 


Communicate candidly 


2.26 


.82 


1.45 


.5692 


State ideas clearly 


1.48 


.20 


.52 


.6839 


Accept responsibility for their own communication behavior 


1.62 


2.94 


.74 


.02()2^- 


Be aware of language indicating bias on gender, ago. 
ethnic, or sexual/affectional orientation 


2.03 


5.70 


1.78 


.0429* 


SKILLS — ROUND 2 


Accept responsibility for their own communication behavior 


1.85 


.23 


.57 


.6701 


Be aware of language indicating bias on gender, age, 
ethnic, or sexual/affectional orientation 


1.48 


.81 


1.08 


.4710 


* Significant differences noted in TUKEY and Least Squi ^eans 



'^able 20b. Disagreements about Basic Speech Communication Skills — Genera! 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Accept responsibility for their own 
communication behavior 

Standard Deviations 


1.75 
.91 


1.47 
.72 


1.88 
1.11 


n.s. 


n.s. 


.0382 


Be aware of language indicating 
bias on gendei, age» ethnic, or 
sexual/affectional orientation 

Standard Deviations 


1.83 
K33 


1.97 
1.19 


2.50 
1.69 


n.s. 


.0428 


n.s. 


EMF = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 
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Table 21a. Analysis of Variance — Message Development and Organization 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr >F 


SKILLS — ROUND 1 


Choose appropriate and effective organizing methods for 
message 


2.50 


.23 


1.53 


850"^ 


Structure a message for effectiveness with an introduction, 
main points, useful transitions, and a conclusion 


2.10 




1 . Z< 1 


.Uw)0 1 


Use summary statement(s) in appropriate contexts 


275 


2.13 


1.47 


.2359 


Outline the key points and sub-points of their spoken message 


2.80 


1.06 


1.93 


.5778 


Choose topic and message about which they are 
comfortable and knowledgeable to speak 


2.90 


32.52 


2.50 


.0001* 


Identify their communication goals 


2.50 


12.63 


1.84 


.0013* 


Accomplish their communication goals 


2.47 


5.55 


1.80 


.0479* 


Select the most appropriate and effective medium for 
communicating 


2.61 


4.69 


1.60 


.0553* 


Develop and present an interesting and attention-getting 
introduction in a speech 


2.96 


16.00 


2.53 


.0022* 


SKILLS — ROUND 2 


Accomplish their communication goals 


2.86 


1.66 


1. 18 


.2494 


Select the mo^t appropriate and effective medium for 
communicating 


2.86 


1.49 


1.12 


.2666 


Choose topic and message about which they are 
comfortable and knowledgeable to speak 


2.97 


4.30 


1.16 


.0268* 


Identify their communication goals 


2.60 


4.20 


1.11 


.0249* 


Develop :md present an interesting and attention-getting 
introduction in a speech 


3.05 


5.08 


1.64 


.0481* 


* Significant differences noted in TUKEY and Least Square Means 
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Several authors address identification and attainment of communication goals including Aitken 
and Neer (1992), Boileau (1982), Camevale et al. (1990), and Wiemann (1977a and 1977b). 
Faculty rated the importance of identifying communication goals significantly higher than did 
employers and policymakers. Faculty also rated the accomplishment of these goals as 
significantly more irucortant than did employers. The selection of the most appropriate 
medium for communication was rated most highly by professors. Faculty members ranked 
significantly more important than did either of the other two groups the development and 
presentation of interesting and attention-getting introductions. 

After the second round of the Delphi instrument, respondents agreed that college students 
should accomplish their communication goals and select appropriate mediums (see Table 21a). 
Comments by faculty for the first statement ranged from emphatic endorsement as "high 
academic priority" to a more mediated characterization. A professor comments, 
"communication ought not to be excessively goal oriented. It serves many nonpurposive 
functions as well." Employers tended to believe the accomplishment of goals is difficult and 
that this skill would develop with experience in the workplace. The emphasis on experience 
continued with the employers' comments on the selection of an appropriate medium. As one 
employer writes, "They [college graduates] may seek guidance for this matter with the 
management." Another administrator remarks, "This is something that can be learned and 
refined over time after seeing needs of environment." Many faculty elaborated on the relevance 
of the skill: "It is important to recognize that different media are available and can be utilized 
according to the topic and context" but others, as well as some employers, suggested the 
choice is not always under the control of the speaker. 

Even after the second round of surveys, the thre-. respondent groups did not agree about the 
importance of choosing a topic about which the communicator is cc fortable and 
knowledgeable to speak, identifying communication goals, and developing and presenting an 
interesting introduction (see Table 21b). There was enough reconciliation between the faculty 
and the policymakers to make the differences no longer statistically significant for the ability 
to choose a topic. However, employers continued to rate this skill as significantly less 
important than did faculty and policymakers. A professor explains, "One of the primary 
reasons there is so much apprehension in speaking is because people do not understand how 
important it is to use topics in their expertise area." The comments by employers stressed the 
absence of topic choice which dominated the speaking situations they most often encounter. 
For example, one writes, "often [it is] not as much [of a choice] as a mandate, (i.e.. -et 
comfortable!)," and another comments, "You don't always have the chance to choose in 
business." There was an emphasis that within the business context, employees have few 
choices to miike in selecting topics. Instead, they are often told or required to deliver certain 
messages. 

Faculty continued to rate goal identification significantly higher than did employers or 
policymakers. Faculty responded with very positive statements. A professor remarks, "Goals 
are crucial in a planned communication event." Employers were less convinced of its 
importance. For example, one employer considered the statement as "academic jargon— not 
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Table 21b. Disagreements about Message Development and Organization 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND I 


Choose to^ ic and message about 
which they are comfortable and 
knowledgeable to speak 

Standard Deviations 


3.76 
1.85 


2.44 
1.50 


3.05 
1.39 


.0001 


.0383 


.0231 


Identify their communication 














goals 

Standard Deviations 


2.95 
1.68 


2.19 
1.15 


2.79 
1.40 


.0036 


n.s. 


.0184 


Accomplish their communication 
goals 

Standard Deviations 


2.78 
1.57 


2.27 
1.24 


2.64 
1.29 


.0366 


n.s. 


n.s. 


Select the most appropriate and 
effective medium for communicating 
Standard Deviations 


2.89 
1.41 


2.42 
1.19 


2.75 
1.26 


.0342 


n.s. 


n.s. 


Develop and present an interesting 
and attention-getting introduction 
in a speech 

Standard Deviations 


3.43 
1.95 


2.61 
1.44 


3.35 
1.48 


.0073 


n.s. 


.0076 


SKILLS — ROUND 2 


Choose topic and message about 
which they aie comfortable and 
knowledgeable to speak 

Standard Deviations 


3.35 
I.I7 


2.86 
1.01 


2.76 
1.13 


.0228 


.0299 


n.s. 


Identify their communication goals 
Standard Deviations 


2.83 
1.06 


2.39 
1.07 


2.86 

1. 00 


.0297 


n.s. 


.0264 


Develop and present an interesting 
and attention-getting introduction 
in a speech 

Standard Deviations 


3.47 
1.68 


2.89 
1.19 


2.94 
.87 


.0476 


n.s. 


p s. 


EMP = Employer: FAC = Faculty; PM = Policymaker: n.s. - not significant 
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[the] real world," and another felt that it was a moot point as "goals will be/can be established 
and identified for them [college graduates]." 

Faculty and policymakers sufficiently integrated their perspectives about the relative merit of 
"attention-getting" introductions such that there was no longer any statistical significance 
between their responses, although faculty rated this skill significantly higher than did 
employers. Faculty members tended to characterize introductions as critical to the 
effectiveness of a message. For example, "You lose people many times in the first few minutes 
of a presentation— sometimes [you can either] build [or] blow [your] credibility for a whole 
speech encounter [during the introduction]," emphasizes a professor. Policymakers valued the 
skill less than did the faculty, stressing the content as more important than stmcture. The issues 
for employers seemed to be applicability and, like the policymakers, the emphasis of content over 
form. As one employer writes, "Entry-level hires do not have much oppormnity to use these skills." 



B.i. c. Basic Speech Communication Skills - Content and Situation Analysis 

The adaptability of speakers to a variety of communication roles is stressed by many authors 
including Aitken and Neer (1992), Backlund et al. (1982), Boileau (1982), and Hanna (1978). 
In this section, the three respondent groups disagreed about the imponance of all six skill 
statements after the first survey (see Table 22a). Faculty and policymakers rated an 
understanding of one's role in a variety of settings (e.g., featured speaker vs. the host who 
introduces featured speaker) as significantly more important than did the employers. 

For the remaining statements in this subsection, faculty rated them significantly higher than 
did employers and policymakers. These skills included preparing a message and adapting the 
communication style to the context and situation as well as the physical setting; adapting to 
changes in audience characteristics (e.g., size, interests, concerns, heterogeneity); choosing 
" and broadening topics according to the needs of the audience; and choosing and narrowing the 
topic according to the occasion. 

By the second round of the Delphi instrument, the respondents reached a consensus about the 
importance of four skills (see Table 22a). The most important skill was adapting to changes in 
the audience characteristics followed by choosing and narrowing a topic as appropriate, 
choosing and broadening the topic, and preparing and adapting a message to changes in the 
physical setting. Faculty stressed the importance of adapting to changes. For example, one 
faculty member writes, 'This part of the process— adaptation— is critical." Another responds 
with "effective communication is based on audience adaptation." Employers and policymakers 
tended to agree with the idea expressed by one policymaker: "I believe these are skills one 
should continue to develop after college. 1 would not expect them to be very refined in these 
areas by the time they graduate." 

Concerning the statement about narrowing of a topic, the comments ranged from considering 
it as a lower level, basic skill to be learned with experience to citing the problems students 
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Table 22a. Analysis of Variance — Context and Situation Analysis 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Understand their roles in a variety of settings 


3.03 


17.18 


2.87 


.0030* 


Prepare a message and adapt communication style to the 
context and situation in which the oral communication occurs 


2.38 


22.75 


1.85 


.0001* 


Adapt to changes in audience characteristics 


2.60 


25.81 


1.79 


.0001* 


Choose and broaden a topic according to the needs of the 
audience 


2.88 


11.80 


2.35 


.0073* 


Choose and narrow a topic as appropriate according to the 
occasion 


2.80 


16.15 


2.17 


.0008* 


Prepare a message and adapt or make changes to the 
physical setting 


3.41 


21.52 


2.93 


.0008* 


SKILLS — ROUND 2 


Adapt to changes in audience characteristics 


2.74 


2.08 


.98 


.1245 


Choose and broaden a topic according to the needs of the 
audience 


2.95 


1.94 


1.09 


.1718 


Choose and narrow a topic as appropriate according to the 
occasion 


2.89 


1.77 


1.05 


.1898 


Prepare a message and adapt or make changes to the 
physical setting 


3.86 


2.52 


1.57 


.2026 


Understand their roles in a variety of settings 


3.05 


6.42 


L26 


.0073* 


Prepare a message and adapt communication style to the 
context and situation in which the oral communication occurs 


2.56 


3.83 


1.01 


.0245* 


* Significant differences noted in TUKEY and Least Square Means 



have actually achieving this goal. Faculty comments about broadening a topic stressed the 
importance of audience. For example, one writes, **Adapting to audience needs is 
fundamental/' In contrast, the comments by policymakers and employers focused on the 
attainment of the skill with experience. One policymaker comments, 'This is an art learned 
from experience.'' An employer adds, ^'[Graduates] should show growth with experience.'* 



Comments about the final statement of this subsection concerning adaptation to physical 
environment varied substantially. One faculty member emphasized its importance writing, 
"Physical setting is as important as content." In contrast, some respondents rated this skill 
lower since they believe speakers often have little control over the physical setting. For 
example one employer writes, "'We have a competent staff to assist us with this." A ^ 
policymaker comments, "Circumstances may not allow for controlling these things." 

After the second survey round, disagreements remained for just two of the six skills (see Table 
22b). The comparative position of faculty and policymakers reversed for the statement 
concerning the importance of understanding communication roles in a variety of settings. In 
fact, policymakers re-evaluated it as more important than the faculty initially did and 
significantly higher than did employers. In the comments about this skill, faculty tended to 
consider it as fairly elementary. As one writes, it is a "simple lesson of great practical 
importance to work." Most policymakers referred to the example in the statement (e.g.. 
featured speaker vs. the host who introduces the featured speaker). "A common fault of the 
introducer is to believe that he/she is the program" states a policymaker. Again the issue for 
employers seemed to be applicability. As one writes, "Again, [there is] very little occasion for 
this. But when it occurs, it is important'^ 

Faculty rated the importance of adapting the communication style to the context and situation 
as significantly more important than did employers. The comments made by respondents about 
this statement tended to fall into three basic categories regardless of their classification (e.g., as 
faculty). Some felt this is a fundamentally basic skJll, that it is more appropriate to associate 
and expect its attainment in high school. Others felt that it was an issue of obtaining more 
work experience. Some, particularly but not exclusively faculty, emphatically stressed its 
importance. A professor writes, "All have a significant impact on success of the message and 
need attention if a speaker is to be even moderately effective." 



B.i.d. Basic Speech Communicafion Skills — Message Support 

Aitken and Neer (1992), Bassett et al. (1978), Hirokawaand Pace (1983), Hunsaker (1989), 
Muchmore and Calvin (1983), and Rubin (1982) discuss reasoning skills that are necessary to 
support a message. In this section, respondents agreed on the importance of ten of the 15 skills 
(see Table 23a). The statement rated as most important was the critical thinking ability of 
recognizing and using basic reasoning [e.g., drawing specific conclusions from general 
information, extrapolating general conclusions from specific information]. The second most 
important statement about the identification of facts, issues, and problems relevant to the topic 
also relates to crifical thinking skills. The respondents also agreed about the importance of 
dcmonstraUng credibility and substantiation of idea.s. 

Some communication scholars (including but not limited to, Backlund et al., 1982; Bassett et 
al., 1978; and Johnson & Szxzupakiewicz. 1987) discuss the skills relating to the 
substantiation of ideas. Of these skills, stating intentions and purposes when appropriate was 
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Table 22b. Disagreements about Context and Situation Analysis 





Means 


Significance Lev'^l 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 




Understand their roles in a variety 
of settings 

Standard Deviations 


3.70 
1.87 


2.74 
1.68 


2,98 
1.46 


.0017 


.0365 


n.s. 


Prepare a message and adapt 
communication style to the context 
and situation in which oral 
communication occurs 

Standard Deviations 


3.09 
1.79 


1.99 
1.20 


2.53 
1.09 


.0001 


n.s. 


.0112 


Adapt to changes in audience 
characteristics 

Standard Deviations 


3.22 
1.80 


2.16 
1.10 


3.05 
1.22 


.0001 


n.s. 


.0001 


Choose and broaden a topic 
according to the needs of the audience 
Standard Deviations 


3.50 
1.98 


2.57 
1.38 


3.18 
1.24 


.0177 


n.s 


.0122 


Choose and naaow a topic as 
appropriate according to the occasion 
Standard Deviations 


3.33 
2.01 


2.45 
1.26 


3.08 
1.16 


.0041 


n.s. 


.0056 


Prepare a message and adapt or 
make changes to the physical setting 
Standard Deviations 


4.09 
2.23 


3.03 
1.51 


3.58 
1.41 


.0021 


n.s. 


.0416 


SKILLS — ROUND 2 


Understand their roles in a variety 
of settings 

Standard Deviations 


3.39 
1.56 


3.08 
.99 


2.57 
.78 


n.s. 


.0045 


.0035 


Prepare a message and adapt 
communication style to the context 
and situation in which oral 
communication occurs 

Standard Deviations 


2.91 
1.34 


2.39 
.90 


2.57 
.74 


.0264 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty: PM = Policymaker; n.s. = not significant 
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Table 23a. Analysis of Variance — Message Support 



Mean 



Mean 
Square 

Model 



Mean 
Square 
Error 



Pr>F 



SKILLS — ROUND 1 



Identify facts, issues, and problems relevant to the topic 


1.96 


2.78 


.94 


.0537 


Demonstrate competence and comfort with information 


2.25 


.74 


1.12 


.5188 


Demonstrate credibility 


2.16 


1.91 


1.88 


.2021 


State intentions and purposes when appropriate 


232 


.51 


1.20 


.6544 


Incorporate information from a variety of sources to 
support messa<;e 


2.68 


6.39 


2.29 


.0638 


Identify and use appropriate statistics to support the message 


2.88 


1-60 


2.47 


.5230 


Recognize and be able to use basic reasoning 


1.88 


.23 


,91 


.7752 


Incorporate language that captures and maintains audience 
interest in message 


2.60 


5.03 


1.78 


,0613 


Use humor when appropriate 


3.29 


.98 


3.05 


.7242 


Use stories and anecdotes when appropriate 


3.50 


4.72 


3.19 


,2305 


Provide appropriate supporting material based on audience, 
occasion, and purpose 


2.30 


11.75 


1.41 


.0003* 


Use motivational appeals that build on values, expectations, 
and needs of the audience 


2.87 


11.77 


2.25 


.0061* 


Research effectively information required for message 
preparation 


2.01 


10.10 


1.64 


.0025* 


Support message by incorporating statements of others into 
their own statements 


2.90 


9.49 


2.17 


.0137* 


Support arguments with relevant and adequate evidence 


1.92 


5.08 


.90 


.0040* 



SKILLS — ROUND 2 



Use motivational appeals that build on values, expectations, 
and needs of the audience 



continued on next page 
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Table 23a. Analysis of Variance — Continued 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


Research effectively information required for message 
preparation 


2.04 


.46 


.72 


.5300 


Support ;irguments with relevant and adequate evidence 


1.92 


.70 


.49 


.2389 


Provide appropriate supporting material based on audience, 
occasion, and purpose 


2.40 


4.09 


1.09 


.0254* 


Support message by incorporating statements of others into 
their own statements 


2.79 


3.17 


1.05 


.0522* 


* Significant differences noted in TUKEY and Least Square Means 



the most highly rated. It was followed by the incorporation of information from a variety of 
sources to support message. A subdivision of this skill is the identification and use of 
appropriate statistics. A consensus was reached for the remaining two skills relating to the use 
of humor and stories and anecdotes. These skills had lower means than did the rest in this 
section on message support. This perhaps reflects the belief that a more advanced level of 
proficiency must be attained before the skills can be integrated successfully into a speaking 
style. Duran (1992) as well as Spitzberg and Hurt (1987) discuss the effectiveness of humor as 
a tool to diffuse social tension, relieve an individual's anxiety, and resolve conflict. 

For all five of the contested statements, faculty ranked the skills higher than did employers or 
policymakers (see Table 23b). The first statement, "College graduates should be able to 
provide appropriate supporting material based on audience, occasion, and purpose'' relates to 
several issues previously discussed, including adaptability and the accommodation of the 
audience. There was a significant difference between the response of faculty and both 
employers and policymakers. For the next skill concerning the use of motivational appeals 
building on values, expectations, and the needs of the audience, policymakers marked ii 
significantly lower in importance than did both employers and faculty. Employers and 
policymakers rated the skill of supporting arguments with relevant and adequate evidence as 
significantly less important than did faculty. 

The ability to rese^ifch information required for message preparation (e.g., seeking information 
in books and periodicals, identifying and questioning appropriate authorities) relates to some 
of the previously mentioned skills concerning the development of supporting material for 
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Table 23b. Disagreements about Message Support 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Provide appropriate supporting 
material based on audience, 
occasion, and purpose 

Standard Deviation 


2.65 
1.39 


1.99 
1.10 


2.68 
1.12 


.0026 


n.s. 


.0014 


Use motivational appeals that build 
on values, expectations, and needs 
of the Jiiidience 

Standard Deviation 


2.78 
1.55 


2.67 
1.48 


3.55 
1.50 


n.s 


.0172 


.0021 


Research effectively information 
required for message preparation 

Standard Deviation 


2.47 
1.78 


1.75 
.94 


2.13 
1.30 


.0061 


n.s. 


n.s. 


Support messatr*". by incorporating 
statements of others into their 
own statements 

Standard Deviation 


3.36 
1.69 


2.65 
1.41 


2.95 
1.30 


.0083 


n.s. 


n.s. 


Support arguments with relevant 
and adequate evidence 

Standard Deviation 


2.09 
1.22 


1.73 
.76 


2.26 
.99 


.0482 


n.s. 


.0039 


SKILLS — ROUND 2 


Provide appropriate supporting 
material based on audience, 
occasion, and purpose 

Standard Deviation 


2.71 
1.12 


2.20 
1.05 


. 2.54 
.92 


.0169 


n.s. 


n.s. 


Support message by incorporating 
statements of others into their own 
statements 

Standard Deviation 


3.10 
1.20 


2.63 
1.00 


2.86 
.84 


.0333 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



messages. Faculty rated this significantly higher than did employers. In addition, faculty and 
employers disagreed about the importance of supporting messages through the incorporation 
of statements developed by others (e.g., quoting correctly or paraphrasing objectively). 
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By the second round of the survey, the respondents had reached consensus for an additional 
three skills in the subsection on message support (See Table 23a). They agreed that college 
graduates should be able to research infornnation effectively, use motivational appeals, and 
support arguments with relevant evidence. The comments by faculty about supporting a 
message through effective research characterized it as a fundamental trait of education and 
central in message preparation: '^Inaccurate or dated information makes presentation useless 
and, sometimes, dangerous." Employers contextualized the skill within their specific fields, 'if 
a recent grad[uate] knows nothing else — he's at least expected to do research correctly," states 
an employer. Another notes "new employees need time to learn industry specific sources and 
technological tools and data bases." However, an executive questions "Who has time to do all 
these things — it's a great theory, but not real world." 

The respondents offered comments about supporting an argument with relevant evidence. An 
employer characterizes it is as a "basic skill. How can you convince others without support ing 
documents?" Faculty members tended to agree that "Ethical and effective use of evidence is a 
key element of effective communication." The most negative of the comments comes from a 
policymaker — "people will believe what you say." 

After the second round of surveys, the respondents failed to agree about the importance of two 
skills: providing appropriate supporting material based on audience, occasion, and purpose and 
supporting messages with the incorporation of statements by others (see Table 23b). Faculty 
continued to rate both of these skills as more important than did employers. Faculty considered 
providing appropriate supporting material as essential. One professor remarks, "This is the real 
base of good communication." Policymakers viewed it as a more sophisticated skill and 
slightly less critical. "This facility comes with post college experience — most often" states a 
policymaker. Another comments, "No real departure here — I just don't think it's that 
important." Employers questioned the value of this skill. As one employer notes, this is "not 
really applicable to entry-level job responsibilities." Employers and policymakers tended to 
view this skill as too advanced for college graduates to achieve. 

Some faculty members stressed the importance of incorporating statements of others into 
messages. "This is basic ethics. Ethical communication necessitates this" stresses one 
professor. A policymaker simply qualified the statement as "manners" while an employer 
considered it as "critical to supporting main idea." 



B .i.e. Basic Speech Communication Skills — Message Type 

Of the five skills in this section, respondents only agreed and rated of medium significance the 
importance of composing and delivering an entertainment-oriented speech when appropriate 
(e.g., an after dinner speech, a toast) (see Table 24a). The respondents rated it much lower than 
they rated most items. 
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Table 24a. Analysis of Variance — Message Type 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILL— ROUND 1 


Compose and deliver an entertainment-oriented speech 
when appropriate 


4,90 


6.35 


4.19 




Compose and deliver an informative speech 


2,17 


24.41 


1.86 


.0001* 


Develop messages that intluence attitudes, beliefs, 
and actions 


2.58 


21.85 


2.14 


.0001* 


Deliver an impromptu or extemporaneous talk about topics 
with which the speiiker is familiar 


3,52 


13.24 


3.48 


.0238* 


Describe or express feelings to others when appropriate 


2,75 


8.31 


2.48 


.0371* 


SKILLS — ROUND 2 


Develop messages that influence attitudes, beliefs, and 
actions 


2.68 


1.55 


.94 


,1949 


Deliver an impromptu or extemporaneous talk about topics 
with which the speaker is familiar 


3.65 


4.22 


L84 


,1043' 


Describe or express feelings to others Vv^hen appropriate 


3.02 


1.41 


1.02 


.2529 


Compose and deliver an informative speech 


2.67 


4.02 


L24 


.0419* 


* Significant differences noted in TUKEY and Least Square Means 



The respondents disagreed about four skills, three of which deal explicitly with different types 
of messages — informative, persuasive, and extemporaneous. Both faculty and policymakers 
rated the composition and delivery of an informative speech significantly higher than did 
employers (see Table 24b). Faculty also ranked the ability to develop messages that influence 
attitudes, beliefs, and actions as significantly more important than did both employers and 
pohcymakers. The statement concerning the delivery of an impromptu or extemporaneous talk 
was rated significantly higher by the policymakers than by the employers. 

The final skill of this section has less to do with the formalized and even formulaic message 
types previously discussed. Many authors (including but not limited to Boileau, 1982 and Duran, 
1983) discuss skills relating to the description or expression of feelings to others. Contrary to 
previous patterns, employers rated this skill significantly highe; i did policymakers. 
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Table 24b. Disagreements about Message Type 





. Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Compose and deliver an informative 
speech 

btandard Deviauon 


2.95 
2.05 


!.80 
1.00 


2.18 
1.06 


.0002 


.0193 


n.s. 


Develop messages that influence 
altitudes, beliefs, and actions 

Standard Deviauon 


2.96 
1.97 


2.17 
1.27 


3.23 
1.12 


.0082 


n.s. 


.0001 


Deliver an impromptu or 
extemporaneous talk about topics 
with which the speaker is familiar 

Standard Deviation 


4.04 
2.32 


3.22 
1.68 


3.70 
1.65 


n.s. 


.0099 


n.s. 


Describe or express feelings to 
others when appropriate 

Standard Deviation 


2.44 
1.27 


2.71 
1.67 


3.28 
1.68 


n.s. 


.0099 


n.s. 


SKILLS — ROUND 2 


Compose and deliver an 
informative speech 

Standard Deviation 


2.65 
1.56 


2.11 
.90 


2.23 
1.00 


.0474 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



After the second round of the survey, consensus was reached for three of the four initially 
contended skills in the message type subgroup (see Table 24a). Of the three, the development 
of messages that influence attitudes, beliefs, and actions was rated the most important. 
Comments by faculty members stressed the applicability of persuasion. A professor notes it is 
the "most important skill for health and growth of society*' while another stresses "Students 
can't only have ideas; they must sell them.'' Employers also emphasized the importance of 
influencing ideas, although they tended to be more specific about the context. For example, 
one writes, "This is selling! We all do it, and it is important that we do it well." Policymakers 
emphasized content or substance rather than the type of message. For example, an 
administrator remarks "the message is more important than the form or style." 

Rerpondents also stressed the importance of delivering an impromptu or extemporaneous talk. 
The following comments by two employers tend to summarize the general perspective of all 
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three groups. "[This skill is] more important to me since impromptu is real world," states a 
manager while another comments, "Employees must always be prepared to respond 
extemporaneously or in an impromptu setting/' The statement about describing or expressing 
feelings to others was considered important. Employers tended to reflect on the skill's impact 
on successful professional development. As one employer explains, "If employees cani 
clearly and concisely express their feelings, issues important to their career may be lost or be 
of less value." In general, comments by faculty tended to be positive, as one qualifies it is 
"important for personal and societal health." 

Composition and delivery of an informative speech was the only skill of this subgroup for 
which consensus was not achieved in the second round of the Delphi instrument (see Table 
24b). Faculty rated this skill significantly higher than did employers. One professor 
characterized the skill as the "most common formal speaking situation." One policymaker 
comments (with some faculty members agreeing) that "persuasion skills [are] vital, 
information [skills are] less so." The general view captured and explained by one employer is 
that "informal communication is more important for [the] entry level employee.'^ Employers 
did not believe that composing and delivering an informative speech was relevant for most of 
their employees. 

From all of the subsections of the Basic Speech Communication Skills, the most valued 
statement for both rounds was the ability to state ideas clearly. The least valued statement was 
the ability to compose and deliver an entertainment-oriented speech when appropriate (e.g., an 
after dinner speech, a toast). 



B.ii. Interpersonal and Group Communication 

This next major category examines the importance of skills relating to the development and 
management of human relations. Central elements include social interaction (including 
conversation), conflict resolution, and small group discussion. While historically much of the 
literature generated by speech scholars jand researchers has focused on the development of 
move formal public messages, there is now much greater awareness of the unique constraints 
that interpersonal and group communication situations pose. Consequently, more attention has 
been placed on studying and researching interpersonal and group interactions. This section is 
subdivided into five basic groups — situation analysis, relationship management, information 
exchange, conversation management, and group communication. 

B.ii.a. Interpersonal and Group Communication — Situation Analysis 

Faculty, employers, and policymakers agreed about the importance of seven skills during the 
first round of the survey (see Table 25a). The most important skill was the ability to identify 
and manage misunderstandings closely followed by recognizing when it is inappropriate to 
speak. The respondents also agreed that college graduates should understand status and 
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Table 25a* Analysis of Variance — Interpersonal and Group Communication — 

Situation Analysis 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Recognize when it is inappropriate to speak 


2.06 


.41 


1.50 


.7604 


Identify and manage misunderstandings 


2.01 


2.46 


1.12 


.1127 


Understand status and relationship between communicators 


3.34 


.37 


2.80 


.8770 


Recognize time constraints of a communication situation 
and know how to operate within them 


2..^0 


.02 


1.47 


.9866 


Adjust to factors that might inhibit effective communication 


2.74 


.33 


1.94 


.8453 


Reduce barriers or interference that may inhibit the 
communication process 


2.92 


7.18 


2.40 


.0523 


Recognize when it is appropriate to schedule or panicipate 
in meetings 


3.19 


.85 


2.92 


.7473 


Recognize when another does not understand their message 


1.80 


1.98 


.74 


.0724* 


Identify and adapt to the perceived needs and desires of 
other communicators 


2.55 


14.77 


1.70 


.0002* 


Make effective decisions during communication situations 


2.26 


5.19 


1.24 


.0166* 


Understand the influence of culture on language 


2.39 


11.98 


1.70 


.0011* 


SKILLS — ROUND 2 


Recognize when another does not understand their message 


1.98 


.06 


.73 


.9181 


Identify and adapt to the perceived needs and desires of 
other communicators 


2.68 


.28 


1.07 


.7707 


Make effective decisions during communication situations 


2.21 


.43 


.76 


.5695 


Understand the influence of culture on language 


2.73 


7.80 


.91 


.0003* 


* Significant differences noted in TUKEY and Least Square Means 
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relationships between communicators (e.g., know when to use formal titles such as "Doctor*' 
or Your Honor**); recognize time constraints of a communication situation and know how to 
operate within them; adjust to factors that might inhibit effective communication (e.g., defer 
questions if time is inadequate, move away from crowds when conducting a serious 
conversation); adjust to factors that might inhibit effective communication; and recognize 
when it is appropriate to schedule or participate in a meeting. 

For the remaining four statements, faculty rated them significantly more important than did 
policymakers (see Table 25b). The contended statements were the abilities to recognize when 
another does not understand their message; to identify and adapt to the perceived needs of 
other communicators in which policymakers disagreed with employers; to make effective 
decisions during communication situations; and to understand the influence of culture on 
language. For this final statement, faculty also disagreed with employers. 

After the second round of the Delphi instrument, consensus was achieved for three additional 
skills: College graduates should be able to recognize the miscomprehension of a message, 
identify and adapt to perceived needs and desires of other communicators, and finally, make 
effective decisions (see Table 25a). 

However, the three groups disagreed about the importance of understanding the influence of 
culture on language (see Table 25b). Faculty rated this skill significantly higher than did 
employers and policymakers. Most respondents tended to write supportive comments for this 
particular skill. A professor remarks, "It is increasingly important to understand the variances 
between cultures in gaining an understanding of individuals.** Another professor states, "The 
context of one*s culture greatly affects meaning.** Other faculty members called for an 
increased emphasis on the development of this skill in college. A professor laments that "too 
few recognize that culture matters to the degree that it really does.** Some policymakers and 
employers wrote similar positive comments and stressed the importance of this skill given our 
increasingly diverse society and the emergence of a more global, international world of 
communication. 

All three respondent groups agreed about the importance of the majority of skills in this 
situation analysis section. The most important skill was the ability to identify and manage 
misunderstandings while the least important was the recognition of when it is appropriate to 
schedule or participate in a meeting. 

B.ii .b. Interpersonal and Group Communication — Relationship Management 

Interpersonal and group communication competence requires relationship management skills. 
These skills include the general development and maintenance of relations, appropriate self- 
disclosure, as well as responding to challenging communication situations. All of these 
components of effective relationship management are discussed throughout the discipline of 
communication studies (e.g., Camevale et al., 1990; Hunsaker, 1989; Larson et al., 1978; 
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Table 25b. Disagreements about Interpersonal and Group Communication Skills — 

Situation Analysis 





Means 


Significance Level 


EMP 


FAC 


PM 


HMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND I 


Recognize when another doevS not 
underj;tand their message 

Standard Deviation 


1.76 
.88 


1.71 
.82 


2.08 
.27 


n.s. 


n.s. 


.0351 


Identify and adapt to the perceived 
needs and desires of other 
communicators 

Standard Deviation 


2.51 
1.46 


2.30 
1.21 


3.30 
1.32 


n.s. 


.0073 


.0001 


Make effective decisions during 
communication situations 

Standard Deviation 


2.33 
1.16 


2.08 
1.05 


2.67 


n.s. 


n.s. 


.0096 


Understand the influence of culture 
on language 

Standard Deviation 


2.80 
1.75 


2.09 
1.13 


2.73 
1.04 


.0081 


n.s. 


.0016 


SKILLS — ROUND 2 


Understand the influence of culture 
on language 

Standard Deviation 


3.17 
1.14 


2.46 
.90 


2.91 
.82 


.0008 


n.s. 


.0088 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



Ruben, 1976; and Stanley & Shockley-Zalabak, 1985). They emphasize lAie importance of 
understanding and valuing differences in communication styles (e.g., speech that is difficult to 
understand due to a handicap, the culture, or accent of the speaker). 

Self disclosure, another element of effective relationship management, is discussed by several 
authors in a variety of different contexts. For example, Bassett et al. (1978), Glaser (1983), and 
Rubin (1982) believe college graduates should be able to express their feelings to others. 
Muchmore and Galvin (1983) discuss the importance of suppressing feelings when their 
disclosure would be inappropriate. Furthermore, Aitken and Neer (1992), Canary and 
Spitzberg (1987), Duran (1992), Glaser (1983), Spitzberg and Hurt (1987), and Sypher (1984) 
emphasize that college graduates should be able to accurately recognize and control levels of 
their own disclosure. 
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The development of relationships through comnnunication is often problematic. Several 
authors describe those challenges and solutions which they require, Di vSalvo and Backus 
(1981) discuss of the identification and management of miscommunication as well as 
Muchmore and Galvin (1983) and wStanlty and Shockley-Zalabak (1985) who review the 
challenges with conflict management in general. Radiffe and Hudson (1987) elaborate on the 
responses to difficult communication situations such as using effective self-assertion skills. 
Another important skill to resolve conflict effectively is empathy. The ability to convey a 
feeling of connection and affinity is cited in the literature (Aitken & Neer, 1992; Cegala, 1981; 
Di Salvo, Larsen, & Seiler, 1976; Duran, 1992; Muchmore & Galvin, 1983; Murphy & Jenks, 
1982; Pai-ks, 1985; Rubin, 1982; Sypher, 1984; and Wiemann, 1977a and 1977b) 

After the initial round of surveys, consensus was reached for only two of the 1 1 skills 
involving relationship management (see Table 26a). The ability to break off non-constructive 
relationships when appropriate was ranked most important followed closely by the ability to 
disclose appropriate information about themselves when relevant. 

For the nine remaining skills there was some disagreement between the three groups (see 
Table 26b). Faculty did not always rank the skills higher than did the other two respondent 
groups as was evident in the other sections. In fact, employers often considered relationship 
maiiagement skills more important than did the other two groups. Employers rated the skills of 
building and maintaining constructive relationships and coping with negative feedback 
significantly higher than did faculty and policymakers. Employers also rated expressing 
feelings to others, effectively asserting themselves, managing conflict, and motivating others 
to disclose information when appropriate significantly higher than did policymakers. However, 
faculty considered both the understanding and valuing differences in communication styles 
along with allov/ing others to express different views as significantly more important than did 
either of the other respondent groups. 

After the second round of surveys, the respondents reached consensus regarding six additional 
skills — understanding and valuing different communication styles, allowing others to express 
different views, describing or expressing feeling to oti^ers, asserting themselves effectively, 
managing conflict, and conveying empathy (see Table 26a). The statement considered most 
important for both rounds of the surveys for this subsection concerned the management of 
conflict. One employer comments, ^Things can really get out of hand in (the) workplace. 
Many conflicts would be avoided if people learned to manage conflict." Some of the faculty 
and policymakers qualified the importance of conflict management. "Again — must balance 
with other concerns (ethical issues, consistency, etc.)'' states a professor. ''Every decision we 
make involves conflict . You have to manage it or it manages you. This is a required skill," 
emphasizes a policymaker. In addition, the respondents agreed that the ability to understand 
and value differences in communication styles is important. An employer states "again, 
successful, mature business people must be sensitive to these issues to be successful. [There is] 
too much opportunity to offend others." "In a diverse workplace, failure here could have grave 
consequences" cautions a faculty member. "Our culture is becoming increasingly diverse. We 
must make every effort to allow for these differences," stresses a policymaker. The respondents 
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Table 26a. Analysis of Variance — 



Relationship Management 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Break-off non-constructive relationships when appropriate 






? Q7 


61 16 


Disclose appropriate information about themselves when 
relevant 


3.28 


5.53 


2.77 


.1384 


Build and maintain constructive relationships with 
superiors, peers, and subordinates 


1.79 


5.60 


.93 


.0028* 


Understand and value differences in communication styles 


2.44 


13.07 


1.90 


.0013* 


Allow others to express different views 


1.65 


4.46 


.86 


.0062* 


Describe or express feelings to others when appropriate 


2.47 


3.84 


1.95 


.1431* 


Cope with negative feedback 


1.94 


5.28 


.91 


.0035* 


Effectively assert themselves 


2.34 


3.67 


1 .35 


.0691* 


Manage contlict 


1.83 


4.03 


1.04 


.0223* 


Convey empathy when communicating 


2.48 


5. .37 


1.61 


.0378* 


Motivate others to disclose information when appropriate 


3.11 


6.87 


2.18 


.0447* 


SKILLS — ROUND 2 


Understand and value differences in communication styles 


2.70 


.18 


.83 


.8050 


Allow others to express different views 


1.86 


.08 


.52 


.8547 


Describe or express feelings to others when appropriate 


2.39 


.28 


1.03 


.7603 


Effectively assert themselves 


2.08 


.53 


.58 


.4046 


Manage contlict 


1.85 


.47 


.58 


.4469 


Convey empathy when communicating 


2.67 


.01 


1.23 


.9895 


Build and maintain constructive relationships with 
superiors, peers, and subordinates 


2.01 


3.08 


.57 


.0055* 


Cope with negative feedback 


2.03 


■ 2.30 


.73 


.04^^8* 


Motivate others to disclose information when appropriate 


3.01 


4.38 


1.42 


.0486* 


* Significant differences noted in TUKEY and Least Square Means 
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Table 26b, Disagreements about Relationship Management 







Means 






nificance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FACPM 


SKILLS — ROUND 1 


Build and maintain constructive 
relationships with superiors, peers, 
and subordinates 

Standard Deviation 


1.42 
.71 


1.88 
1.04 


2.05 
1.04 


.0034 


.0015 


n.s. 


Understand and value differences 
in communication styles 

Standard Deviation 


2.91 

1.80 


2. 12 
1.22 


2.70 
1.11 


.0048 


n.s. 


.0074 


Allow others to express different 
views 

Standard Deviation 


1.85 
1.61 


1.46 
.80 


1.90 
.90 


.0259 


n.s. 


.0081 


Describe or express feelings to 
others when appropriate 

Standard Deviation 


2.25 
1J6 


2.44 
1.48 


2 83 
1.47 


n.s. 


.0449 


n.s. 


Cope with negative feedback 

Standard Deviation 


1.56 
.83 


2.09 
1.02 


2.03 
.89 


.0005 


.0123 


n.s. 


Effectively assert themselves 

Standard Deviation 


2.05 
1.19 


2.38 
1.16 


2.60 
1.28 


n.s. 


.0256 


n.s. 


Manage conflict 

Standard Deviation 


1.56 
.94 


1,85 
1.01 


2.15 
1.14 


n.s. 


.0097 


n,s. 


Convey empathy when 
communicating 

Standard Deviation 


2.27 
1.27 


2.42 
1.30 


2.93 
1.16 


n.s. 


.0110 


.0241 


Motivate others to disclose 
information when appropriate 

Standard Deviation 


2.73 
1.38 


3.17 
1.56 


3.48 
1.34 


n.s. 


.0095 


n.s. 


SKILLS — ROUND 2 


Build and maintain constructive 
relationships with superiors, peers, 
and subordinates 

Standard Deviation 


1.68 
.89 


2.08 
.72 


2.20 
.68 


.0143 


.0050 


n.s. 


continued on next page 
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Table 26b. Continued 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


Cope with negative feedback 

Standard Deviation 


1.75 
1.01 


2.16 
.74 


2.03 
.92 


.0248 


n.s. 


n.s. 


Motivate others to disclose 
information when appropriate 

Standard Deviation 


2.64 
1.40 


3.20 
1.24 


2.94 


.0348 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



agreed that the related skill of allowing others to express different views was important as well 
as expressing oneself in an assertive nmanner. The respondents' comments about effective 
assertion stressed a distinction between assertion and aggression. One employer notes that 
assertion should not become aggressive or domineering." "Assertion, not aggression is essential^ 
concurs a professor. A policymaker concludes, "If you are going to assert, make it effective." 

Employers considered conveying empathy as fundamental to their specific jobs. ''[It's] very- 
important to H[uman] R[esources]" states an executive while another manager stresses **[It's] 
very important in customer- related fields." An employer elaborates, "Much of our work involves 
interactions, and we, therefore, must be compassionate and understanding of others feelings." 
Policymakers further expanded on the importance of this skill. An administrator writes, "A high 
level of empathy enhances the probability of a good communicator." Faculty, in general concurred, 
but one professor made this distinction, "I'm less concerned with conveying empathy than their 
ability to understand and take into account the position of another pei-son." 

For the three remaining skills for which consensus was not reached, employers consistently 
rated them higher than did the policymakers and faculty (see Table 26b). Employers 
emphasized the importance of building and maintaining constructive relationships. Employers 
commented that relationship building is "absolutely essential," and its absence is "a major 
cause of lost jobs." Policymakers believed this skill increases with e/.pe .ence. One professor 
notes that this is an **admirable goal, but [it] goes beyond speech competency and requires 
experience graduates don't yet possess." Most faculty considered it as a skill that would be 
developed after completion of college. 

The ability to cope with negative feedback was significantly more important from the 
employer perspective than from the faculty perspecfive. Despite this difference, some faculty 
and policymakers tended to be positive in their comments about the skill. One faculty member 
even went so far as to say that there is "no such thing as negative feedback." However, 
respondents from all three groups did offer some qualifiers. One employer found that 
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personnel evaluations did not always result in improvement. In addition, one policymaker 
stressed the importance of both the source and the purpose of the feedback. 

For the skill of motivating others to disclose information, employers rated it significantly more 
important than did faculty. Employers emphasized situations in which this skill was important 
to them. One writes, *This is an essential aspect of [an] auditor's job.'' Another employer 
comments, '*A11 of these are critical skills for those who work in schools — be they teachers or 
support staff." Faculty tended to be concerned with the ethical implications of this. To some, 
the statement suggested a level of manipulation or coercion with which they were 
uncomfortable. For example, one professor notes, '^Disclosure ought to be voluntary." *This 
item suggests [self-disclosure] should be 'motivated' by another," remarked another. A third 
refeired to George Orwell's novel, "[this skill statement] sounds 1984-ish." 

B.ii.c. Interpersonal and Group Communication — Information Exchange 

Information exchange is considered CJi important skill by many communication experts. Curtis, 
Winsor, and Stevens (1989), Di Salvo and Backus (1981), and Ratliffe and Hudson (1987) discuss 
the importance of giving feedback appropriately and the ability to offer constructive criticism. 
Rubin (1982) notes the importance of giving concise and accurate directions. 

From the initial survey, the respondents reached consensus for five of the eight skills in this 
subsection (see Table 27a). The three skills ranked as most important were listening attentively 
to questions and comments from other communicators, asking questions effectively, and 
answering questions concisely and to the point or issue. Respondents also agreed about the 
importance of giving feedback appropriately and giving concise and accurate directions. 

A pattern of disagreement emerged for three skills in this section. Employers and faculty felt 
responding appropriately to feedback was significantly more important than did poHcymakers. 
Employers also considered college students ability to ask questions when they do not 
understand another's message as well as their ability to paraphrase or restate what a speaker 
has said to confirm his/her meaning of greater importance than did the policymakers. 

After the second round of the survey, the respondents reached consensus on all of the skills in 
which they differed in round one. They agreed that college graduates should respond 
appropriately to feedback, ask quesfions when another's message is unclear, and restate a 
message to confirm meaning. 

B.ii.d. Interpersonal and Group Communication — Conversation Management 

The importance of initiating and managing conversations is discussed by Ratliffe and Hudson 
(1987) and Spitzberg and Hurt (1987). Di Salvo and Backus (1981) and Glaser (1983) 
reviewed the importance of interviewing skills. 
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Table 27a. Analysis of Variance 



— Information Exchange 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Listen attentively to questions and comments from other 
communicators 


1.76 


1.43 


.85 


.1894 


Give feedback appropriately 


2.18 


1.42 


1.07 


.2690 


Ask questions effectively 


1.82 


1.57 


.79 


.1402 


Answer questions concisely and to the point or issue 


l97 


.55 


.93 


.5537 


Give concise and accurate directions 


2.24 


.17 


1.85 


.9103 


Respond appropriately to feedback 


1.98 


2.25 


.87 


.0765* 


Ask questions when they do not understand another's 
message 


1.95 


3.84 


.96 


.0194* 


Paraphrase or restate what speaker has said to confirm 
his/her meaning 


2.57 


3.48 


1.60 


.1163* 


SKILLS — ROUND 2 


Respond appropriately to feedback 


2.03 


1.74 


.89 


.1455 


Ask questions when they do not understand another's 
message 


1.93 


1.50 


.76 


.1426 


Paraphrase or restate what speaker has said to confirm 
his/her meaning 


2.86 


1.84 


1.21 


.2215 


* Significant differences noted in TUKEY and Least Square Means 



Among the 13 skills in this subsection, respondents agreed about the importance of nine areas 
in the first round (see Table 28a). The means for this section illustrated that respondents tended 
to rank these skills as less important than those other sections of this survey. The three most 
valued skills were college graduates' ability to be open-minded about another's point of view, 
manage multiple conversational goals effectively, and convey enthusiasm for the topics 
through delivery. Respondents also reached consensus about the importance of introducing 
new topics, sustaining topics and discussion and offering follow-up comments, interrupting 
effectively, beginning and ending a conversation effectively, negotiating effectively, giving and 
receiving compliments gracefully, and giving bad news to others with empathy. 
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Table 27b. Disagreements about Information Exchange 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Respond appropriately to feedback 
Standard Deviation 


1.87 
.92 


1.92 
.93 


2.28 
.93 


n.s. 


.0403 


.0428 


Ask questions when they do not 
understand another's message 

Standard Deviation 


1.73 
.87 


.1.93 
.99 


2.30 
1.07 


n.s. 


.0068 


n.s. 


Paraphrase or restate what speaker 
has said to confirm his/her meaning 
Standard Deviation 


2.33 
1.37 


2.58 
1.27 


2.88 
1.09 


n.s. 


.0331 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



For the remaining four skills, there were different perceptions (see Table 28b), Faculty and 
employers ranked the ability to demonstrate attentiveness ttirough non-verbal and verbal 
behaviors (e.g., nod, respond with facial expressions to message, vocalize agreement by saying 
"yes" or "uh-huh") significantly more important than did policymakers. Faculty rated 
significantly higher than did policymakers the ability to convey enthusiasm for the topic 
through delivery (e.g., vary pitch volume, and tone; use appropriate stance, posture, eye 
contact to develop a rapport with the audience). The ability to approach and engage in 
conversation with new people in new settings was important in the literature (for example, 
Duran, 1992). It was ranked significantly more important by employers than by faculty and 
policymakers. 

After the second survey round, the respondents reached agreement for all four skills. College 
graduates should convey enthusiasm for the topic through delivery, have confidence to 
approach and engage in conversation, be open-minded, and demonstrate their attentiveness 
(see Table 28a). The abilities of conveying enthusiasm and open-mindedness were ranked by 
respondents as more important than any of the skills for which consensus was reached in the 
initial round. Some of the comments about conveying enthusiasm for a topic through delivery 
suggested that such a skill relates to more formal or "public" speaking. As one employer notes, 
"New employees do litde public speaking." However, some respondents from each group 
considered it as a skill developed by mature adults with work experience. A professor states, 
"This skill is refined with experience, not book learning." "This increases with experience — 
don't expect fullness," agrees a policymaker. 
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Table 28a. Analysis of Variance — Conversation Management 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND I 


Manage nnultiple communication goals effectively 


2.41 


.40 


1.74 


.7930 


Begin and end a conversation in an appropriate and 
socially acceptable manner 


2.74 


.05 


2.26 


.9793 


Introduce new topics when appropriate 


3.27 


2.06 


2.14 


.3834 


Sustain topics and discussion and offer follow-up 
comments when appropriate 


3.12 


.55 


1.96 


.7565 


Interrupt effectively when appropriate and possible 


3.27 


1.01 


2.47 


.6657 


Conduct and participate in an interview 


2.61 


.36 


2.32 


.8554 


Negotiate effectively 


2.79 


.66 


2.81 


.7901 


Give bad news to others with empathy 


2.90 


4.33 


2.65 


.1984 


Give and receive compliments gracefully 


3.09 


.86 


3.37 


.7750 


Dennonstrate attentiveness through nonverbal and verbal 
behaviors 


2.64 


9.37 


2.14 


.0139* 


Convey enthusiasnn for topic through delivery 


2.38 


4.31 


1.69 


.0810* 


Have confidence to approach and engage in conversation 
with new people in new settings 


2.56 


7.24 


1.77 


.0183* 


Be open-minded about another's point of view 


1.71 


6.95 


.91 


.0007* 


SKILLS — ROUND 2 


Demonstrate attentiveness through nonverbal and verbal 
behaviors 


2.93 


3.88 


1.64 


.0966 


Convey enthusiasm for topic through delivery 


2.30 


.18 


1.27 


.8683 


Have confidence lo approach and engage in conversation 
with new people in new settings 


2.65 


1.38 


.91 


.2203 


Be open-minded about another's point of view 


2.19 


.37 


.87 


.6559 


* Significant differences noted in TUKEY and Least Square Means 
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Table 28b. Disagreements about Conversation Management 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND \ 


Demonstrate attenti\eness through 
nonverbal and verbai behaviors 

Standard Deviation 


2.53 
1.60 


2.47 
1.36 


3.25 
1.55 


n.s. 


.0293 


.0066 


Convey enthusiasm for topic 
through delivery 

Standard Deviation 


2.29 
1.23 


2.28 
1.35 


2.80 
1.24 


n.s. 


n.s. 


.0347 


Have confidence to approach and 
engage in conversation with new 
people in new settings 

Standard Deviation 


2.13 
1.23 


2.67 
1.37 


2.83 
1.34 


.0110 


.0113 


n.s. 


Be open-minded about another's 
point of view 

Standard Deviation 


1.24 


1.50 
.78 


2.15 

-97 


n.s. 


n.s. 


.0003 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



The skill about having confidence to approach people also garnered positive comments. As 
one employer explains, "There is so much business that takes place in a social setting that this 
is a critical component to a successful businessperson/* One faculty member relates it to the 
mission of his/her college: "Lack of confidence may truly hinder lifelong learning — the goal 
for any self-respecting, liberal arts college/' 

The comments from each of the respondent groups stressed the importance of open- 
mindedness. According to one employer, it is "essential to communication. Listening with an 
open mind is essential for team building [in a matrix environment] and for interpersonal 
relationship building.'' A policymaker explains, "This doesn't mean we will agree with the 
other person, but we must consider his viewpoint. We can't assume we are the bearers of all 
wisdom." One faculty member considers this skill as an expected byproduct of a college 
education: "[It is] the single most important contribution college can make to a person's 
development." 

Despite respondents' lower rating of the importance of demonstrating attentiveness skill in the 
second round, the faculty comments were generally positive. For example, one writes that this 
skill "creates an atmosphere that generates more information." Some faculty did qualify the 
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importance of the skill. As one professor explains, it "indicates superficial listening only.'' The 
comments by employers maintained a similar pattern of endorsement. One employer 
emphasizes that '*Body language is everything! If for no other reason than to effectively 
interview." Another expresses concern that "attentiveness may be perceived as 'fake' if over 
emphasized by listener." Corresponding to their slightly lower evaluations, some policymakers 
tended to react to the skill in more negative terms than did faculty or employers. For example, 
one policymaker characterizes demonstrating attentiveness as an "annoying behavior, [it] should be 
discouraged.*' Another considers the skill "over-emphasized — [and that it] can be deceptive " 

B.ii.e. Interpersonal and Group Communication — Group Communication 

Curtis. Winsor and M.evens (1989), Di Salvo (1980), Di Salvo et al. (1976), Hanna (1978), 
Hirokawa and Pace (1983), Lohr (1974), and Muchmore and Galvin (1983) and others discuss 
group problem solving skills. Many of these were discussed in other sections, such as 
appropriate levels of self-disclosure; identification of facts, issues, and problems relevant to 
the topic; and being open-minded about the views of others contribute to effective group 
problem solving. Other relevant abilities include the ability to brainstorm or initiate ideas, to 
request information efficiently, and to evaluate ideas carefully. 

In this final subsection of interpersonal communication, the respondents agreed about the 
importance of all 15 skills (see Table 29). The most important skill was working on 
collaborative projects as a team followed by the ability to motivate others to participate and 
work effectively as a team. Respondents also agreed that understanding and implementing 
different methods of building consensus; fostering a sense of community between group 
members; identifying points of agreement and disagreement between communicators; 
identifying common purposes and interests of a group; matching people, interests, and tasks 
when coordinating a group project; recognize all members' role in a small group (e.g., 
recognize when they are acting as a leader, facilitator, disrupter); allowing others to take credit 
for achievement when appropriate; generating conversation and discussion from quite 
members in a group; managing troublesome members in a group; confronting others 
effectively in appropriate contexts; leading meetings effectively, keeping group discussions 
relevant and focused, and setting and managing realisiic agendas were all important skills. 

B.iii. Communication Codes 

Communication codes are generally considered to be important skills for college students. The 
abilities to use and understand spoken English including the use of pronunciation, grammar, 
and articulation appropriate to the designated audience are cited by communication scholars as 
essential skills (Backlund et al., 1982; Bassett et al., 1978; Duran, 1983; Hymes, 1986; 
Johnson & Szczupakiewicz, 1987; McCroskey, 1982; Morreale, 1990; Rubin, 1982; Sypher, 
1984). Equally important considerations in communication codes are the ability to use 
appropriate nonverbal behaviors such as gestures or facial expressions (Duran, 1989; 
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Table 29. Analysis of Variance — Group Communication 





Mean 


Mean 
Square 
Mode! 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Work on collaborative projects as a teana 


1.80 


1.42 


1.12 


.2829 


Motivate others to participate and work effectively as a team 


2.26 


.66 


1.85 


.7011 


Understand and implement different methods of building 
consensus 


2.44 


1.74 


1.80 


.3813 


Foster a sense of community between group members 


2.71 


1.85 


2.08 


.4134 


Identify points of agreement and disagreement between 
communicators 


2.59 


2.21 


1.68 


.2699 


Identify common purpose and interests of a group 


2.53 


2.05 


1.76 


.3143 


Match people, interests, and tasks when coordinating a 
group project 


2.98 


.00 


2.37 


.9996 


Recognize all members' roles in a small group 


3.05 


1.29 


3.09 


.6591 


Allow others to take credit for achievement when appropriate 


2.37 


1.08 


1.98 


.5803 


Generate conversation and discussion from quiet members 
of a group 


3.15 


3.59 


2.50 


.2398 


Manage troublesome members in a group 


2.97 


3.55 


2.54 


.2497 


Confront others effectively in appropriate contexts 


2.86 


.30 


2.14 


.8678 


Set and manage realistic agendas 


2.63 


1.84 


2.06 


.4117 


Lead meetings effectively 


2.80 


4.63 


2.53 


.1635 


Keep group discussions relevant and focused 


2.52 


4.54 


1.95 


.0998 


* Significant differences noted in TUKEY and Least Square Means 



Muchmore & Galvin, 1983; Parks, 1985; Ruben, 1976; Rubin, 1982; Spitzberg & Hurt, 1987; 
andWiemann, 1977a, 1977b). 



From a total of nine skills in this section, the respondents initially agreed on the importance of 
six abilities (see Table 30a). The most important skill was the use of pronunciation, grammar, 
and articulation appropriate to the designated audience. The next two skills were rated equally 
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Table 30a. Analysis of Variance — Communication Codes 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Use pronunciation, grammar, and articulation appropriate 
to the desif^nated niidipnrf* 




1 1 ^ 


1 "^Q 
i ,J J 




Use appropriate vocal behaviors for the message and the 

fiiidipncp 








.4VV/0 


Use visual and other aids effectively to support ideas. 

motivntp nnH npr^iindf* nthf*r<; 




1 Q 






^nf niihlipiv r\r in Ponvf^rcntion'iJ cf»fHn{TC n/ithmif 

O^^tXN ^Ul/llvLj' KJl 111 v\Jll VC-l C^clllVJIlui oClltll^d WllllV^Ul 

displaying extreme anxiety or nervousness 


2,40 


2.13 


1.66 


.2781 


Focus without fear on speaker and message 


2.59 


1.81 


1.73 


.3533 


Adant to iinnnticinntpd chjinrrpc in fhp <cpftincr in whirh 

iWU^t K\J Ul IcUJ i 1 V l^UlWU Vlldll^U-i^ 111 C>L>lllll^ 111 WIIIVll 

communication takes place 


3.07 


5.69 


2.48 


.1033 


Use nnnronrintf nnnvprhiil hphiiuinr^i for fh^* m*^<M!irrA imH 

VJO^ 1 (11^ IIVIIVC^I (Jell UC^lldVlV^lD IWl lliC ltl\r?>>^cl^C cll Ivi 

the audience 


2.40 


9.86 


1.80 


.0048* 


Identify and respond appropriately to discrepancies 
between the speaker*s verbal and nonverbal message 


3.01 


5.00 


1.94 


.0784* 


Recognize nonverbal behaviors and respond appropriately 


2.80 


8.44 


2.13 


.0204* 


SKILLS — ROUND 2 










Use appropriate nonverbal behaviors for the message and 
the audience 


2.84 


l.Ol 


1.21 


.4345 


Identify and respond appropriately to discrepancies 
between the speaker's verbal and nonverbal message 


3.03 


2.69 


1.24 


.1170 


Recognize nonverbal behaviors and respond appropriately 


2.85 


2.30 


1.32 


.1789 


* Significant differences noted in TUKEY and Least Square Means 



important by the respondents. College graduates should use appropriate vocal behaviors for 
the message and audience. They should also speak publicly or in conversational settings 
without displaying extreme anxiety or nervousness. The remaining skills rated as important 
were using visual and other aids effectively to support ideas, motivate and persuade others 
along with focusing without fear on the message, and adapting to changes in the setting. 
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Faculty, employers, and policymakers disagreed about the importance of three skills (see Table 
30b). The use of appropriate nonverbal behaviors was rated significantly more important by 
faculty and employers than by policymakers. Faculty rated significantly higher than did 
policymakers the abilities to recognize nonverbal behaviors and discrepancies between the 
verbal and nonverbal message. However, by the second round of surveys, the three groups 
agreed about the importance of ^/// skills in this communication codes section (see Table 30a). 
Faculty particularly underscored the importance of using appropriate nonverbal behaviors. A 
professor notes. "Conservatively 60-65 percent of what we communicate is through nonverbal 
channels." "Nonverbal [communication] carries more weight and is more significant for 
listeners" according to an instructor. Another faculty member states that being attune to 
nonverbal behaviors is especially important for "intercultural encounters." A professor 
cautions that sensitivity to nonverbal [behaviors] is extremely important, but I'm afraid we 
may oversimplify its complexity by encouraging easy recognition." 

In summary, after two rounds of surveys, the respondent groups agreed about the importance 
of using and understanding both spoken English and nonverbal signs or cues as well. The most 
important skill in this section was the use of pronunciation, grammar, and articulation 
appropriate to the designated audience. The lowest-ranked skill, although still considered 
important, was the ability to adapt to changes in the setting. 

B.iv. Oral iMessage Evaluation 

iMany of the previous sections which presented important speech communication skills 
focused on the expressive abilities of college graduates. However, these graduates are also 
receivers of messages. College graduates' abilities to receive, process, and evaluate 
information from the environment and from other people are major factors that contribute to 
their ability to communicate effectively. 

College graduates are surrounded by numerous stimuli in the workplace and as citizens in 
society. Their ability to select certain stimuli to use as information is necessaiy for successful 
communication. When reception occurs and an individual attends to the message or acts upon 
the information, then communication between people has occurred. 

Listening is a significant communicative event (Lai'son et al., 1978). A considerable amount of time 
is spent in receiving messages rather than producing them. The listener has some control over the 
success of the message transfer betv^^een individuals as does the sender. Competent communication 
can be viewed as a "mutual process, engaged in by both the sender and the receiver, who share the 
responsibility for the creation of meaning in the interaction" (Larson et al., 1978, p. 48). 

The way a person listens to another will affect the type of relationship that will develop 
between the listener and the speaker (Egau, 1 970). An individual who listens in a superficial, 
closed or critical way will develop a very different relationship than a person who listens 
closely with an open mind is attentive and supportive lo the speaker (Larson et af, 1978). 
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Table 30b. Disagreements about Communication Codes 





Means 


Sig 


nificance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Use appropriate nonverbal behaviors 
for the message and the audience 

Standard Deviation 


2.41 
1.35 


2.18 
1.31 


3.00 
1.40 


n.s. 


.0424 


.0020 


Identify and respond appropriately 
to discrepancies between the 
speaker's verbal and nonverbal 
message 

Standard Deviation 


3.09 
1.38 


2.83 
1.44 


3.40 
1.26 


n.s. 


n.s. 


.0209 


Recognize nonverbal behaviors and 
respond appropriately 

Standard Deviation 


2.83 
1.49 


2.60 
1.49 


3.35 
I.3I 


n.s. 


n.s. 


.0035 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = 


not significant 







At the most basic level, listening requires that the speaker and other interactants understand 
words and phrases (Backlund et al., 1982; Bassett et al., 1978; Rubin, 1981, 1982). Listeners 
interpret messages on the basis of their own cognitive processes, goals, and needs. When 
accurate interpretation takes place, there is a common language between the speaker and 
listener. Listening is an active process where the receiver of the message selects various 
portions to seriously attend to or act upon. Since listening is a skill, it can be learned or 
improved upon in the educational process. 

Listening also requires individuals to concentrate or understand, and effecuvely evaluate 
messages (Bienvenu, 1 97 1 ; Camevale et al., 1990; Cegala, 1981; Curtis et al., 1989; Di Salvo, 
1980 and 1981; Di Salvo et al., 1976; Hanna, 1978; Hunsaker, 1989; Johnson & 
Szczupakiewicz, 1987; Rubin. 1984; Stanley & Shockley-Zalabak, 1985; Sypher, 1984; 
Witkin, 1973). In order to make judgments about messages, listeners need to use their critical 
thinking abilities to reach decisions. Effective listeners search for main ideas and critical 
supporting points, develop a sense of empathy and an awareness of biases both on the part of 
the speaker and listener, and decide which parts of the message to concentrate on and retain as 
well as which to discard (Hunsaker, 1989; Larson et al., 1978; Muchmore & Galvin, 1983). 

Effective listeners respond to more than vocal cues. Listening involves the reception of data 
through many senses. Therefore, listening includes the ability of the receiver to respond to 
nonverbal cues as well. 
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In the final section of this instrument, faculty members, employers, and policymakers rated the 
importance of a variety of message evaluation skills. This section consisted of listening 
behaviors identified from the relevant literature discussed above. Basic levels of 
comprehension were explored as well as more sophisticated levels of critical thinking. 

After the first round of surveys, the respondents reached a consensus about the importance of 
eight skills. They agreed that the two most important skills were the ability to listen attentively 
and with an open mind (see Table 31a). The next most valued skill was distinguishing facts 
from opinions. This latter result is consistent with the importance of reasoning as discussed by 
numerous scholars (e.g., Aitken & Neer. 1992; Bassett et al. 1978; Hirokav>^a & Pace, 1983; 
Hunsaker, 1989; and Rubin, 1982). 

Faculty, employers, and policymakers also considered the abilities to identify important points 
when given oral instructions, to evaluate evidence, and to distinguish main points from 
supporting details as essential skills for college graduates. The lowest rated, yet important 
skill, was the ability to recognize sarcasm and irony. 

Respondents disagreed about the importance of the six remaining skills. In all cases, faculty 
rated these skills as significantly more important than did employers (see Table 31b). Many of 
these skills were critical thinking abilities such as the analysis of assumptions, evidence, and 
conclusions; the detection and evaluation of bias or prejudice; and the evaluation of speeches 
on the basis of credibility of both the message and the speaker Faculty disagreed with 
policymakers on two of the items—listening empathetically to help speakers clarify their 
thoughts and listening carefully to speakers with accents or impairments. In both cases, the 
faculty gave high ratings. 

By the second round of surveys, the three groups of respondents reached an agreement about 
the importance of two additional skills. College graduates should be able to listen 
empathetically to help speakers clarify their thoughts or feelings. They should listen carefully 
to speakers with strongly accented or impaired speech, and they should recognize the 
speaker's purpose or goal. However, they disagreed about the importance of four skills. 

Faculty rated significantly more important than did employers the ability to detect and evaluate 
bias and prejudice. Professors comments that this skill is "one of the primary skills in 
communication especially with the variety of people in the real worid," and this is "important 
for work and everyday life.*' Another instructor wishes that all college graduates could do this 
while another faculty member remarks that recognizing "individual bi?s is central and difficult 
for college graduates, particularly for community college students. One professor states "this 
takes experience and maturity often beyond the college grad years." Another instructor stresses 
that in community colleges, students only begin to learn this skill and it's not appropriate to 
expect them to be competent in this area. A professor concludes that students need more 
exposure and practice on learning this skill and criticizes the textbooks about speech as not 
covering this area. Since the textbooks are deficient in covering this skill, the instructor 
believes it should not be expected of college graduates. Few employers noted their responses 
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Table 31a, Analysis of Variance — Oral Message Evaluation 





Mean 


Mean 
Square 
Model 


Mean 
Square 
F^Tror 


Pr>F 


SKILLS — ROLNI) 1 


Listen lUtentivcly 


1.51 


1.(5 


.72 


.2046 


Listen with an open mind 


1.51 


.70 


.76 


.3997 


Identify important points when given oral instructions 


1.97 


1.^)0 


.Q2 


.1300 


Distinguish main points from supporting details 


2.14 


3.60 


1.61 


.1098 


Distinguish facts from opinions 


1.75 


.20 


1.23 


.8488 


I'A'aluaie v/ays in which speaker's ideas might be applied in 
new or different situations 


- — 

2.17 




2.01 


.3252 


Hvaluate evidence on the basis of relevance, 
appropriateness, and adequacy 


2.1 1 


3 27 


1 .38 


.0970 


Recognize sarcasm and irony 


2.78 


3{)S 


.1 13 


.2381 


Listen empaihetically to help speaker clarify their thoughts 
and feelings 


2.30 


5.00 


1.50 


.0372* 


Analyze assumptions, evidence, and conclusions of an 
argument 


2.12 


6.4K 


1.52 


.0152* 


Detect and evaluate bias and prejudice 


2X)^ 


6.03 


1.36 


.0132* 


Evaluate speeches and messages on the basis of the 
credibility of both the speaker and his/her message 


2.33 


10,77 


2.06 


.0061* 


Listen carefully to speakers with strongly accented or 
impaired speech 


2.36 


7.07 


1.77 


.0200* 


Recognize speakers purpose or goal 


2.01 


5,47 


1.36 


.0194=* 


SKILLS — ROUND 2 


Listen empathetically to help speakers clarify their 
thoughts and feelings 


2.44 


2.65 


.98 


.0704 


Listen carefully to speakers with strongly accented or 
impaired speech 


2.28 


.74 


.95 


.4580 


continued on next pa^e 
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Table 31a. Continued 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


Analyze assumptions, evidence, and conclusions of an 
ari^umcnt 


L93 


3.18 


.65 


.0085* 


Detect and evaluate bias and prejudice 


2.05 


2.77 


.82 


.0360* 


Evaluate speeches and messages on the basis ot* the 
credibility of both the speaker and his/her message 


2.15 


4.95 


.84 


.0034* 


Recognize speakers purpose or goal 


2.01 


1.58 


.82 


.1495* 


* Significant differences noted in TUKEY and Least Squiure Means 



for the ratings they gave to these skills. One employer believes students are becoming sensitive 
to this area and notes the importance of it since "they will have to deal with less enlightened 
custofiiers m the real world." Another employer comments that older eraployeer, who have 
more experience and are established within an organization are better prepared to detect and 
evaluate bias than new employees who are recent graduates. 

Employers rated significantly lower than did policymakers and faculty the ability to analyze 
assumptions, evidence, and conclusions. The respondents wrote mostly positive comments in 
support of the importance of this skill. A professor states, "This is the heart and soul of critical 
thinking." "Critical thinking and listening is a valuable base for an assessment. Reasoning and 
analysis is a fundamental to the discussion" comments another faculty member. Other faculty 
believes this skill is essential to become better consumers, voters, parents, and employees. A 
policymaker also stresses the importance of analysis and questions "what could be more 
important in evaluating the merits of an argument." However, another faculty member believes 
this skill is too advanced for community college graduates and that they will have to learn 
these skills later. Policymakers rated the skill to recognize the speaker's goal as significantly 
more important for college graduates than did employers. 

The final skill in this section where the respondents disagreed about its importance was the 
ability to evaluate speeches on the basis of credibility of both message and speaker. Employers 
rated this skill as significantly less important than did faculty and policymakers in their 
evaluations. A professor stresses this is of the "utmost importance," and another instructor 
comments this is a "high academic priority." The ability to evaluate credibility is "really 
necessary given the number of unethical speakers out there" according to a faculty member. 
Other respondents disagree about its importance. A professor remarks that "credibility is vastly 
overrated." Some employers believe that new employees or recent college graduates do not 

1 12 



i2^ 



Table 31b* Disagreements about Oral Message Evaluation 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND! 


Listen empathetically to help 
speakers clarify their thoughts and 
feelings 

Standard Devkirion 


2 5"* 
1.31 


1.13 


1.34 


.U4vZ 


n.s. 




Analyze assumptions, evidence, and 

conclusions of ;in nroiimf^nt 

Standard Deviation 


1.60 


1 Q\ 
I .V J 

1.00 


Z. 1 J 

1.28 


X) 140 


n.s. 


n.s. 


Detect ;ind evalii;ift* hi'i*: nnrf 
prejudice 

Standard Deviation 


2,43 
1,51 


1.86 
1.02 


1.98 
1.05 


.0143 


n.s. 


n.s. 


Evaluate speeches and messages on 
the basis of the credibility of both 
the speaker and his/her message 

Standard Deviation 


2.85 
Ml 


2.09 
1.35 


2.30 
1.14 


.0062 


n.s. 


n.s. 


Listen carefully to speakers with 
strongly accented or impaired speech 
Standard Deviation 


2,62 
1.52 


2.12 
1,23 


2.68 
1.35 


.0386 


n.s. 


.0255 


Recognize speaker*s purpose or goal 
Standard Deviation 


2,30 
1,^U 


1.81 
.99 


2.23 
1.23 


.0269 


n.s. 


n.s. 


SKILLS — ROUND 2 




Analyze assumptions, evidence, and 
conclusions of an argument 

Standard Deviation 


2.27 
1,41 


1.82 
.65 


1.80 
.68 


.0228 


.0305 


n.s. 


Detect and evduate bias and prejudice 
Standard Deviation 


2,37 
1,16 


1.93 
.88 


1.97 
.57 


.0369 


n,s. 


n.s. 


Evaluate speeches and messages on 
the basis of the credibility of both 
the speaker and his/her message 

Standard Deviation 


2.58 
1,20 


2,03 
.85 


1.94 
.68 


,0125 


.0058 


n,s. 


Recognize speal<er's purpose or goal 
Standard Deviation 


2.17 
1. 00 


2,03 
.93 


1.76 
.70 


n,s. 


,0425 


n,s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n,s, = not significant 
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Table 31c. Reliability of Items in Speech Communication Survey 



Sections 


Number of Items 


rtipud 




Basic speech communication skills 


41 


.9571 




General 


6 


.7096 




Message development/organization 


9 


.8818 




Context and situation analysis 


0 






Message support 


1 c 


Ql 1 
.V 1 1 




Message type 


r 
D 






Interpersonal and group communicaiion 


58 


.9749 




Situation analysis 


11 


.8970 




Relationship management 


11 


.8813 




Information exchange 


o 


.8879 




Conversation management 


13 


.9218 




Group Communication 


1 C 


OS40 




Communication Codes 


9 


.9182 




Oral message evaluation 


14 


.9327 




Total speech communication survey 


122 


.9852 





posses this skill while a few employers do not believe college graduates need this particular 
skill for entry-level positions. 

The evaluations of oral messages and their effects are very important skills according to the 
respondents. They agreed about the importance of ten sub-skills related to listening behaviors 
and evaluation. The two most highly related skills of extreme importance were listening 
attentively and with an open mind. Overall, the respondents believed that many of these 
critical thinking skills are essential for college graduates to become effective listeners in work 
and society. 



B.v. Reliability 

The results of the reliability analysis for the speech cuinmunication goals inventory is 
presented in Table 31c. The interpersonal and group communication section had the highest 
reliability (a = .97) while the communication codes section had the lowest reliability (a = .92). 
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In the basic speech communication section, the subsection of general skills had the lowest 
reliability (a = .71) while the context and situation analysis subsection had the highest 
reliability (a = .93). In the interpersonal and group communication section, the conversation 
management subsection had the highest rehabiUty {a = .95) while the relationship 
management section had the lowest reliability (a = .88). Overall, the reliabiUty of individual 
sections and subsections tended to increase with the number of items comprising a given unit. 
The majority of reliability coefficients were above a = .80. 



B.vi. Factor Analysis 

The items in the speech communication survey were further analyzed by conducting a 
principle components factor analysis with varimax rotation. In the speech area, 12 factors were 
extracted since the original conceptualization consisted of 12 dimensions that were believed to 
account for the speech communication variables. The rotated factor matrix is illustrated in 
Tables 32a through 32e 

Many of the variables from the 'interpersonal and group communication" section loaded most 
heavily on Factor 1. Specifically, the majority of items in the subsections of "relationship and 
conversation management" comprised Factor 1. These skills included disclosing appropriate 
information and describing or expressing feelings when appropriate. The skills related to 
adaptation also were in Factor 1. These adaptive skills included adjusting to time constraints, 
unanticipated changes in the setting, and the needs or desires of other communicators. In 
addition, three of the 1 1 items in the subsection '^situation analysis" appeared in Factor 1. 
These related to the speaker's adaptive skills. None of the skills from the "group 
communication" subsection loaded onto Factor 1. This factor accounted for approximately 14 
percent of the variance. 

Factor 2 accounted for 1 1 percent of the variance. The majority of items in this factor were 
from the ''group conamunication" subsection. These skills related to the college graduates 
ability to manage and develop groups. 

Factor 3 accounted for eight percent of the variance. Two subsections from "basic speech 
communication" skills dominated this factor. The "context and situation analysis" and the 
"message type" were prevalent. These skills included choosing and narrowing or broadening a 
topic, preparing and adapting the message to the particular context, and delivering certain 
kinds of speeches such as an informative presentation. 

Factors 4 through 6 each accounted for seven percent of the variance. Factor 4 consisted of 
listening skills primarily from the "oral message evaluation" section. These skills included 
listening attentively and with an open mind as well as identifying important points when given 
oral instructions. Factor 5 is comprised of mainly the advanced critical thinking abilities 
required to evaluate messages. The abilities to analyze assumptions, evaluate the credibility of 
speakers, distinguish facts from opinions, and detecting bias or prejudice were contained in 
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Table 32a. Speech Goals Survey — Factor Analysis — Factors 1 and 2 



Goals 


Pnptnr 1 


Listen empathetically 


.44886 


Motivate others to disclose information 
when appropriate 




Give concise and accurate directions 


.41637 


.69885 


Conduct and participate in an interview 


.38769 


Interrupt effectively when appropriate 


.68570 


Speak publicly or in conversational 
settings without displaying anxiety 


.36424 


Disclose appropriate information about 
themselves when relevant 




Percent of variance accounted for by factor 


14.32 


.66781 ■ 


Goals 


Factor 2 


Begin and end a conversation in an 
appropriate manner 


.64913 


Understand and implement different 
methods of building consensus 




when appropriate 


.63742 


Motivate others to participate and work 
effectively as a team 


.73935 


Introduce new topics when appropriate 


.62488 


Recognize when it is appropriate to 
schedule/participate in a meeting 


.62447 


Match people, interests, and tasks when 
coordinating a group project 


. / jZ4j 


Adjust to factors that might inhibit 
effective communication 


.61795 


Manage troublesome members in a group 


.09519 


Break-off non-constmciive relationships 
when appropriate 


.61569 


Identify common purpose and interests 
of a group 


.68800 


Generate conversation and discussion 
from quiet group members 


,68098 


Sustain topics and discussion and offer 
follow-up comments 


.60014 


Use humor when appropriate 


.59121 


Lead meetings effectively 


.66806 


Give and receive compliments gracefully 


.57930 


Identify points of agreement and 
disagreement between communicators 


.65454 


Describe or express feelings to others 
when appropriate 


.57281 


Foster a sense of community between 
group members 


.64764 


Paraphrase or restate what speaker has 
said to confirm his/her meaning 


.56309 


Confront others effectively in 
appropriate contexts 


.64435 


Understand status and relationship 
between communicators 


.54584 


Recognize all members' roles in a 


.63412 


Convey empathy when communicating 


.53085 


olllclll ^luup 


Have confidence to approach and 
engage in conversation 


.52541 


Allow others to take credit for 
achievement when appropriate 


.63313 


Adapt to unanticipated changes in the 
setting 


.51826 


Keep group discussions relevant and focused 


,62612 


Set and manage realistic agendas 


.58288 


Recognize and adapt to time constraints 


.51799 


Give bad news to others with empathy 


.50462 


Reduce barriers or interference that may 
inhibit communication 


.49938 


Work on collaborative projects as a team 


.49481 




.43858 


Identify and adapt to the needs or 
desires of other communicators 


.46946 


Build and maintain coastructive relationships 


Percent of variance accounted for by factor 


11.22 
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Table 32b. Speech Goals Survey — Factor Analysis — Factors 3 and 4 



Goals 


Factor 3 


Goal 


Factor 4 


Adapt to changes in audience 
c h a rn ctp ri <i 1 1 f <c 


, /OoO / 


Listen with an open mind 


.75210 


Choose and narrow a topic 


,72685 


Listen attentively 


,67939 


Choose and broaden a topic 


.69348 


Be open-minded about another's point 
of view 


,66301 


Prepare a message and adapt /make 
changes to the physical setting 


.64132 


Allow others to express different views 


,61393 


Deliver an impromptu or 
extemporaneous talk 


.58326 


Recognize speaker's purpose or goal 


.48869 


Prepare and adapt communication 
style to the context and situation 


,58303 


Listen attentively to questions and 
comments 


,47154 


Develop messages that influence 
attitudes, beliefs, and actions 


,56518 


Identify imponant points when 
giving oral instructions 


.40715 


Compose and deliver an 
entertainment-oriented speech 


.51899 


Listen carefully to speakers with 
accented or impaired speech 


,40319 


Understand their roles in a variety 
of settings 


.49787 


Focus without fear on speaker and 
message 


,32321 


Compose and deliver an informative 
speech 


.49595 


Use visual and other aids to support 
ideas, motivate, and persuade others 


,30673 


Develop and present an interesting 
and attention-getting introduction 


,48198 






Use motivational appeals that build 
on values, expectations, and needs 


,43038 






Incorporate language which stimulates 
audience interest 


.41370 






Percent of variance accounted for by factor 


8.28 


Percent of variance accounted for by factor 


7,07 



this factor. Factor 6 consisted of mainly items from the "basic speech communication" skills 
section. These skills included stating ideas clearly, identifying and accomplishing 
communication goals, outlining key points, and using appropriate and effective organizing 
methods for the message. 

Factors 7 and 8 each accounted for six percent of the variance. The majority of the "message 
support'' items loaded onto Factor 7 while many of the "relationship management'' items 
loaded onto Factor 8. Factors 9 through 12 each accounted for less than five percent of the 
variance. 
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Table 32c. Speech Goals Survey — Factor Analysis — Factors 5 and 6 





Factor 5 


Goals ' 


Factor 6 


Detect and evaluate bias and prejudice 


.74313 


Accomplish their communication goals 


.67301 


Evaluate the relevance, adequacy, and 
appropriateness of evidence 


.71045 


Stnjcture a message with an 
introduction, main points, etc. 


.67094 


Analyze assumptions, evidence, 
and conclusions of an argument 


.70714 


Choose appropriate and effective 
organizing methods for message 


.64406 


Evaluate the credibility of speakers and 
their messages 


.64988 


Select the most appropriate and 
effective medium 


.58522 


Distinguish facts from opinions 


.58957 


Identify their communication goals 


.53947 


Evaluate ways in which ideas mighi be 
applied in different situations 


.56488 


Use summary statements in appropriate 
contexts 


.53147 


Distinguish main points from supporting 
details 


.55268 


State ideas clearly 


.47578 


Recognize sarcasm and irony 


.53166 


Communicate candidly 


.47418 






Outline key points and sub-points 


.46763 






Use pronunciation, grammar, and 
articulation appropriate to audience 


.45515 






Recognize when it is appropriate to 
communicate 


.45482 






Choose topic about which they are 
comfortable and knowledgeable 


.42172 


Percent of variance accounted for by factor 


7.07 


Percent of variance accounted for by factor 


7.00 



This factor analysis illustrated that many of the variables loaded onto a factor v/ere originally 
grouped within the survey. Many of the relationship and conversation management skills were 
interrelated and appeared in one factor suggesting that there was not a need for two separate 
subsections. Many of the group communication items loaded onto a single factor while we had 
integrated this area with interpersonal communication. Group communication could be 
considered as a separate topic that warrants further elaboration. 

Twelve factors are not necessary to define speech communication given the results from this 
analysis. Relationships and conversation management could be combined together as well as 
situation analysis and information exchange. 
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Table 32d. Speech Goals Survey — Factor Analysis — Factors 7 and 8 



1 

Goals 


Factor 7 


Goal 


Factor 8 


Support arguments with relevant and 
adequate evidence 


.67921 


Ask questions effectively 


.39146 


Identify facts, issues, and problems 
relevant to the topic 


.63786 


Manage conflict 


.64958 


Research effectively information 
required for message preparation 


.58630 


Identify and use appropriate statistics to 
support the message 


.58155 


Identify and manage misunderstandings 


.55205 


Effectively assert themselves 


.53193 


Incorporate information from a variety 

of sources 


.54497 


Cope with negative feedback 


.51610 


Support message by incorporating 
statements of others 


.52343 


Respond appropriately to feedback 


.50717 


Provide appropriate supporting material 
given the constraints 


.50748 


Give feedback appropriately 


.46494 


Recognize and be able to use basic 
reasoning 


.49984 


Manage multiple communication goals 
effectively 


.46396 


Demonstrate credibility 


.47465 


Recognize when another does not 
understand their message 


.4373 1 


State intentions and purposes when 
appropriate 


.43135 


Negotiate effectively 


.42199 


Demonstrate competence and comfort 
with information 


.40666 


Ask questions when they do not 
understand another's message 


.41092 






Answer questions concisely and to the 
point or issue 


.37^ 27 


Percent of variance accounted for by factor 


6,16 


Percent of variance accounted for by factor 


5.70 



B.vii, Summary 

Faculty, employers, and policymakers judged a large array of speech communication skills. 
After two rounds of surveys, the entire group agreed about the importance of many skills. In 
the areas of information exchange, conversation management, and group communication, the 
three groups reached a consensus about all of these skills. In a similar manner, they also 
agreed about the importance of all skills relating to using and understanding spoken English 
and non-verbal signs considered as ^'communication codes" (see Table 33). 
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Table 32e. Speech Goals Survey — Factor Analysis — Factors 9 through 12 



Goals 


Factor 9 


Goals 


Factor 10 


Use appropriate nonverbal behaviors 


.62871 


Be aware of language indicating bias on i 
age, ethnicity, etc. 


ender, 
.52688 


Convey enthusiasm for topic through 
delivery 


.57675 


Understand the influence of culture on 
language 


.50393 


Use appropriate vocal behaviors 


.53885 


Accept responsibility for their own 
communication behavior 


.45595 


Demonstrate altentiveness through 
nonverbal and verbal behaviors 


.53348 


Understand and value differences in 
communication styles 


.43496 


Recognize nonverbal behaviors and 
respond appropriately 


.50719 


Communicate ethically 


.38506 


Identify and respond to discrepancies 
between verbal and nonverbal message 


.48350 


Make effective decisions during 
communication situations 


.38461 


Percent of variance accounted for by 
factor 


4.48 


Percent of variance accounted for by 
factor 


3.60 


Goals 


Factor 1 1 


Goal 


Factor 12 


Use stories and anecdotes when 1 
appropriate 


.49698 


Recognize when it is in appropriate 
to speak 


.45726 


Percent of variance accounted for 
by factor 


2.56 


Percent of variance accounted for 
by factor 


2.34 



Faculty tended to value the importance of preparing students for both public speaking roles as 
well as interpersonal conimunication roles. Therefore, the faculty rated significantly higher 
than did the employers the skills of composing and delivering an informative speech and 
presenting and interesting, and attention-getting introduction. Employers believed that formal 
public speaking was not relevant for many of their employees in the workplace. However, 
there were certain relationship management skills that employers rated higher than professors. 
These skills included building and maintaining constructive relationships with superiors, peers, 
and subordinates; coping with negative feedback; and motivating others to disclose 
information when appropriate. Employers viewed these skills as being extremely important for 
employees to develop especially within the context of working with teams of diverse 
individuals. 

In terms of the development and organization of a message, policymakers rated significantly 
higher than did faculty and employers the importance of choosing a topic with which the 
speaker is knowledgeable and comfortable. The identification of communication goals by the 
speaker was rated significantly higher by the faculty than the judgements of employers and 
policymakers. Both of these skills require the speaker to have some freedom to make decisions 
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Table 33* Summary of Consensus &. Disagreements in 
Each Section of Speech Communication Survey 



Section of Survey 


Round I — 
Number of Items 


Round 2 ~ 
Number of Items 


Final — 
Number of Items 


A 


% 


D 


% 


A 


% 


D 


% 


A 


% 


D 


% 


Basic Speech 


Communication Skills 


19 


46.3 


22 


53.7 


14 


63.6 


8 


36.4 


33 


80.5 


8 


19.5 


General 


4 


66.7 


2 


33.3 


2 


100.0 


0 


00.0 


6 


lOO.O 


0 


00.0 


Message Development 


























and Organization 


4 


44.4 


5 


55.6 


2 


40.0 


3 


60.0 


6 


66.7 


3 


33.3 


Context and Situation 


























Analysis 


0 


00.0 


6 


100.0 


4 


66.7 


2 


33.3 


4 


66.7 


2 


33.3 


Message Support 


10 


66.7 


5 


33.3 


3 


60.0 


2 


40.0 


13 


86.7 


2 


13.3 


^.'essage Type 


1 


20.0 


4 


80.0 


3 


75.0 


1 


25.0 


4 


80.0 


1 


20.0 


Interpersonal and Group 


Communication 


38 


65.5 


20 


34.5 


16 


80.0 


4 


20.0 


54 


93.1 


4 


16.9 


Situation Analysis 


7 


63.7 


4 


36.3 


3 


75.0 


I 


25.0 


10 


90.9 


1 


9.1 


Relationship Management 


2 


18.2 


9 


81.8 


6 


66.7 


3 


33.3 


8 


72.7 


3 


27.3 


Information Exchange 


5 


62.5 


3 


37.5 


3 


100.0 


0 


00.0 


8 


100.0 


0 


00.0 


Conversation Management 


9 


69.2 


4 


30,8 


4 


100.0 


0 


00.0 


13 


100.0 


0 


00.0 


Group Communication 


15 


100.0 


0 


00.0 










15 


100.0 


0 


00.0 


Communication Codes 


6 


66.7 


3 


33.3 


3 


100.0 


0 


00.0 


9 


100.0 


0 


00.0 


Oral Message Evaluation 


8 


57.1 


6 


42.9 


2 


33.3 


4 


66.7 


10 


71.4 


4 


28.6 


TOTAL 


71 


58.2 


51 


41.8 


35 


68.6 


16 


31.4 


106 


86.9 


16 


13.1 


A = Number of Items for which there was agreement; D = Number of Items for which there was 
disagreement 



about their own presentations. Employers strongly asserted that employees have little control 
over what particular topics they will speak about with their audiences. Often employees are 
told what their communication goals will be and are required to speak about certain topics as 
mandated by their supervisors. In a similar manner, employers believed that college graduates 
only need to understand their roles in the current workplace setting rather than multiple or 
diverse contexts. They also considered the preparation and adaptation of the speaker's style to 
various contexts as less important than did the faculty members. Both policymakers and 
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employers thought that the ability to adapt messages was contingent upon employees gaining 
substantial experience in the workplace. 

The faculty rated significantly higher than did employers and policymakers the skills of 
providing appropriate supporting material based on the audience, occasion, and purpose as 
well as supporting messages by incorporating statements of other into the speaker's 
statements. Faculty viewed these skills as critical to the development of effective 
communicators. However, employers and policymakers thought that both of these skills were 
too advanced to expect from college graduates. 

In terms of the skills associated with effective evaluations of oral messages, there were 
additional disagreements about the importance of four skills. Employers rated significantly 
lower than did faculty and policymakers the ability to analyze assumptions, evidence, and 
conclusions of an argument. They also believed that the skills of detecting bias and prejudice 
were not developed in college as well as recognizing the speaker's purpose or goal and the 
ability to evaluate a message on the basis of the credibility of both the speaker and the actual 
presentation. While employers valued these skills, they believed that these evaluation skills 
were too advanced for college graduates to achieve. There was an expectation that through 
experience in the workplace and in society that employees would become more competent. 

Overall, the three respondent groups agreed about importance of the majority of speech 
communication skills. They only disagreed with about one-tenth of the items. In most cases, 
these disagreements existed because either employers or policymakers believed that certain 
skills were too advanced to expect from college graduates. 



B.vii.b. Advanced Communication Skills 

There were a variety of communication skills that the respondent groups agreed were 
important for college graduates to achieve. These skills ranged from very basic competencies 
to more advanced levels. The respondent groups viewed the basic skills as critical to providing 
a foundation that was necessary to progress to higher levels of achievement. These basic skills 
are the n-iinimal levels of communication functioning that are necessary for college graduates 
to attain. Examples of basic skills include using summary statements, selecting an effective 
medium for communicating, using appropriate pronunciadon, grammar, and articulation, and 
structuring a message with key features such as an introduction, main points, useful 
transitions, and a conclusion. 

Most of the advanced speech communication skills are linked with critical thinking and 
require reasoning skills. In order to communicate effectively, college graduates need to speak 
or listen in ''such a way that certain desirable outcomes are enhanced cr facilitated" (Larson et 
al., 1978, p. 10). While the basic skills may be viewed as competencies necessar, for minimal 
levels of performance, advanced cognitive knowledge is required. College graduates must 
possess both skills and knowledge in order to make decisions about appropriate 
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communication behaviors. These decisions require the speaker or listener to be aware and 
understand the relationships between people involved in the communication. Furthermore, in 
order to detennine what is appropriate in a given situation, college graduates need to understand the 
norms of the culture or the particular organization as well as the group nomis which provide 
guidance about the acceptable usage of language as well as ways to communicate in specific 
relationships. College graduates need to make a wide range of judgements as they prepare fO'tnal 
speeches, communicate in small groups and working teams, or listen to messages. 

College graduates with advanced skills are able to eifectively analyze the situation or context 
within which they must develop their message. They clearly understand their roles and can 
adapt to changes in their audiences. College graduates are also able to consider both their 
audience, the context, and the purpose of their message as they prepare their communication. 

College graduates synthesize information from a variety of sources to support their message 
and to demonstrate credibility by choosing appropriate material to support their arguments or 
claims. They also identify the facts, issues, or problems relevant to the particular topic. College 
graduates employ advanced skills when they use reasoning to draw conclusions from general 
information or extrapolate general conclusions from specific information. Some 
communicators seek to deliver more than infonnation. They may compose and deliver a 
speech or message to influence others' attitudes or beliefs. 

College graduates also need to interact effectively with other people. The development and 
management of human relations requires advanced thinking skills. College graduates with 
advanced abilities identify and adapt to the perceived needs and desires of other 
communicators. They make effective decisions during the communication situation. College 
graduates as new employees build and maintain constructive relationships with their superiors, 
peers, and subordinates in the workplace. They also understand and value differences in 
communication styles. They manage conflict and convey empathy when communicating. As 
college graduates work in groups or teams, they collaborate effectively with others and learn to 
be interdependent. They motivate others to participate and understand different methods of 
building consensus. College graduates who facilitate group communication can foster a sense 
of community between group members and generate conversation from quiet members. 

College graduates are also the receivers of many messages, presentations, or other 
communication acts. In these situations, they employ listening skills that also require 
reasoning to make evaluations about the communications. College graduate with advanced 
skills effectively distinguish facts from opinions and main points from supporting details, They 
evaluate the ways in which the speaker's ideas might be applied in new or different situations. 
They also evaluate evidence on the basis of relevancy, appropriateness, and adequacy. They 
assess the credibility of speaker and the message. 

College graduates who learn or develop basic speech communication skills as well as 
advanced reasoning abilities are effective communicators. They attain their communication 
goals. They can also effectively evaluate messages and effectively interact with other people. 
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Advanced skills in both writing and speech communication require the development of 
reasoning skills. These links with critical thinking have been highlighted in the summaries of 
our findings. We also developed and conducted a separate survey which focuses in more detail 
on critical thinking. 



C. Critical Thinking Skills 

C.i. Interpretation Skills 

Interpretation is considered to be an important ability for college graduates to achieve 
(Facione, 1990; Paul & Nosich, 1991). This skill involves understanding and expressing the 
meaning and significance of a variety of communications. 

C.i.a. Interpretation Skills - Categorizing 

Within this broad category of interpretation, the ability to categorize information is critical 
(Facione, 1990; Marzano et al., 1988). The results from the initial Delphi study indicated there 
was an agreement about all items in the categorization section (see Table 34). All three 
stakeholder groups believed that college graduates should be able to make comparisons by 
noting similarities and differences between or among informational items. This extremely 
important skill was followed by the ability to formulate categories, distinctions, or fmmeworks 
to organize information in such a manner to aid comprehension. The ability to classify and 
group data, findings, and opinions on the basis of their attributes or a given criterion was also 
rated as extremely important. The next skill, translating information from one medium to 
another to aid comprehension without altering the intended meaning, was also rated as 
extremely important. 



C.i.b. Interpretation Skills - Detecting Indirect Persuasion 

Another essential group of abilities are in the area of detecting indirect persuasion. A good 
critical thinker should be able to understand the underiying motives contained or expressed in 
language or other communication systems such as social behaviors. This ability is similar to 
Facione's (1990) "decoding significance" category which we originally used in our survey. 
However, focus group members, in particular, believed Facione's term was an academic term 
that they did not readily understand. Therefore, we changed the term to "detecting indirect 
persuasion." The ideal goal is that students with this skill can detect and describe the content 
and affective intentions and purposes expressed in communication. After the first survey, the 
stakeholder groups disagreed about the importance of every item in this subsection (see Table 
35a). Faculty rated these skills significantly higher than did either the policymaker or 
employer groups. The one exception was in the area of detecting "if, then" statements based 
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Table 34. Analysis of Variance — Interpretation Skills — Categorizing 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS ~ ROUND 1 


Formulate categories, distinctions, or fnirnev/orks lo 
organize information in such a manner to aid comprehension 


2.00 


.66 


1.40 


,6238 


Translate information from one medium to another to aid 
comprehension without altering the intended meaning 


2.54 


2.49 


1.75 


,2434 


Make comparisons; note similarities and differences 
between or among infonnational items 


1.94 


1, 70 


1.09 


.2016 


Classify and group data, fmdings, and opinions on the 
basis of attributes or a given criterion 


2.24 


1. 01 


1.56 


.5239 


* Significant differences noted in TUKEY and Least Square Means 



on a false assumption. The policymakers rated this skill higher than faculty and as significantly 
more important than the employers' evaluations (see Table 35b). 

After the second round of surveys, the three groups reached a consensus about four skills. 
College graduates should be able to detect the use of leading questions that are biased towards 
eliciting a preferred response. They should recognize the use of misleading language such as 
language that exaggerates or downplays the importance of something or neutralizes a 
controversial topic. College graduates should also recognize the use of slanted definitions or 
comparisons which express a bias for or against a position. Furthermore, college graduates 
should detect instances where irrelevant topics or consideration are brought into an argument 
that divert attention from the original issue. These four skills tended to be rated as extremely 
important. 

The respondents disagreed about two remaining skill areas. Faculty rated significantly higher 
than did employers and policymakers the ability to detect the use of strong emotional language 
or imagery which is intended to trigger a response in an audience (see Table 35b). A professor 
states, "Strong emotional language is the basis for political rhetoric. It could lead people 
astray/' Another professor remarks, "One cannot assess reasons unless one can identify 
emotional language and set it aside.'' Other faculty concur and note, "Emotive language and 
imagery are among the pitfalls to effective reasoning,'' and this skill is "necessary for 
evaluating persuasive appeals in politics and advertising.'' Another faculty member in a 
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Table 35a. Analysis of Variance — Interpretation Skills — Detecting Indirect Persuasion 





Mean 


Mean 
Squiu^e 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Detect the use of strong emotional language or imagery 
which is intended to trigger a response in an audience 


2.61 


22.52 


2.60 


.0003* 


Detect the use of leading questions that are biased towards 
eliciting a preferred response 


2.36 


13.20 


2.15 


.0026* 


Detect "if, then" statements based on the false assumption 
that if the antecedent is true, so must be the consequence 


2.41 


5.83 


2.08 


.0637* 


Recognize the use of misleading language 


2.36 


9.27 


2.08 


.0129=^ 


Detect instances where irrelevant topics or considerations 
aie brought into an argument that divert attention from 
the original issue 


2.51 


10.22 


1.79 


.0040* 


Recognize the use of slanted definitions or compiirisons 
v/hich express a bias for or against a position 


2.46 


12.59 


2.09 


.0029* 


SKILLS — ROUND 2 


Detect the use of stiong emotional language or imagery 
which is intended to trigger a response in m audience 


2.58 


8.86 


1.51 


.0036* 


Detect *'if, then" statements based on the false assumption 
that if the antecedent is true, so must be the consequence 


2.34 


2.35 


1.51 


.2150* 


Detect the use of leading questions that are biased towards 
eliciting a preferred response 


2.18 


.89 


.87 


.3615 


Recognize the use of misleading language 


2.21 


.51 


1.01 


.6039 


Detect instances where inelevant topics or considerations 
are brought into an argument that divert attention from 
the original issue 


2.59 


2.07 


1.27 


.1990 


Recognize the use of slanted definitions or comparisons 
which express a bias for or against a position 


2.44 


L34 


.74 


.1695 


* Significant differences noted m TUKEY and Least Square Means 
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Table 35b, Disagreements about Detecting Indirect Persuasion 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Detect the use of strong emotional 
language or imagery which is 
intended to trigger a response in 
an audience 

Standard Deviation 


3.48 
1.76 


2.27 
1.53 


2.67 
1.62 


.0002 


.0377 


n.s. 


Detect the use of leading questions 
that are biased towards eliciting a 
preferred response 

Standard Deviation 


3.05 
1.75 


2.12 
1.32 


2.28 
1.54 


.0029 


.0425 


n.s. 


Detect "if, then'' statements based 
on the false assi'mption that if the 
antecedent is true, so must be the 
consequence 

Standard Deviation 


2.83 
1.65 


2.36 
1.41 


2.08 
1.27 


n.s. 


.0266 


n.s. 


Recognize the use of misleading 
language 

Standard Deviation 


2.93 
1.66 


2.15 
1.41 


2.36 
1.27 


.0089 


n.s. 


n.s. 


Detect instances where irrelevant 
topics or considerations are brought 
into an argument that divert attention 
from the original issue 

Standard Deviation 


3.00 
1.67 


2.25 
1.19 


2.78 
1.35 


.0096 


n.s. 


.0402 


Recognize the use of slanted 
definitions or comparisons which 
express a bias for or against a 
position 

Standard Deviation 


3.12 
1.80 


2.21 
1.31 


2.44 
1.38 


.0041 


n.s. 


n.s. 


SKILLS — ROUND 2 


Detect the use of stron.^ emotional 
language or imagery which is 
intended to trigger a response 
in an audience 

Standard Deviation 


3.12 
1.34 


2.32 
1.23 


2.86 
1.01 


.0038 


n.s. 


.04141 


continued on next page 
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Table 35b. Continued 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


Detect **if, then*' statements based 
on the false assumption that if the 
antecedent is true, so must be the 
consequence 

Standard Deviation 


2.39 
1.00 


2.43 
1.41 


1.90 
.54 


n.s. 


.0237 


.0080 


ExMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



professional field emphasizes, *1 am basing this on my profession and program, occupational 
therapy, wherein working v/ith patients and clients of all ages and disabilities, this is a very 
important skill.'' Some faculty members stress how difficult it is to teach students to recognize 
strong emofional language. Some professors believe it is very difficult for college graduates to 
make such detections. A professor concludes, '^Students don't recognize facts from opinions." 

Other employers and policymakers disagree about the importance of this skill. An employer 
notes, *This kind of persuasion is not normal in business decision making." Although, another 
employer remarks, **As the ocean of new facts and information grows more quickly than our 
ability to assimilate this information, it seems we ai'e more susceptible to 'trigger' mechanisms 
in lieu of having facts to challenges statements with. I think knowing what these triggers are 
becomes increasingly important as the glut of information increases." Several policymakers do 
not consider this skill to be important. Some emphasize the understttnding of content as the 
most important ability. 

Policymakers rated significantly higher than did both faculty and employers the importance of 
detecting *'if, then" statements based on false assumptions (see Table 35b). Several faculty 
members note a difficulty in understanding what this particular item meant. However, some 
faculty advocate its extreme importance. They state this is a "basic form of reasoning" and "as 
our society becomes more legalistic- as it surely is - these abilities become crucial." Another 
professor remarks, "A graduate who has gone through a logic course as I expect, should be 
able to have this skills. These are simple skills but very important for adequate function at this 
level " Another professor counters, "It is nice if people are able to recognize verbal 
manipulation, but not essential in all areas of life and work." A policymaker stresses this skill 
is *'more meaningful to college graduates beyond the two-year associate degree." **In today's 
world of sound bites, playing upon existing assumptions and prejudices can go undetected 
unless people have learned to take the time to examine critically the linkages in messages and 
arguments" srates another policymaker. Overall, faculty view this skill as more basic and 
fundamental, which may in part explain why it received a lower rating. Policymakers view this 
as an important skill that should be developed further in college. 
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Table 36a. Analysis of Variance — Interpretation Skills — Clarifying Meaning 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Recognize confusing, vague or ambiguous language that 
requires clarification to increase comprehension 


2.23 


.81 


1.45 


.5727 


Ask relevant and penetrating questions to clarify facts, 
concepts, and relationships 


1.82 


1.87 


1.33 


.2480 


Identify and seek additional resources, such as resources 
in print, that can help clarify communication 


2.22 


1.53 


1.45 


.3508 


Develop analogies and other forms of comparisons to 
clarify meaning 


2.73 


4.06 


1.88 


.1176 


Recognize contradictions and inconsistencies in written 
or verbal language, data, images, or symbols 


2.01 


4.24 


1.17 


.0288* 


Provide an example that helps to explain something or 
removes a troublesome ambiguity 


2.40 


5.19 


1.38 


.0250* 


SKILLS — ROUND 2 


Provide an example that helps to explain something or 
removes a troublesome ambiguity 


2.05 


.22 


.59 


.6859 


Recognize contradictions and inconsistencies in written 
or verbal language, data, images, or symbols 


1.91 


1.58 


.52 


.0493* 


* Significant differences noted in TUKEY and Least Square Means 



C.i.c. Interpretation Skills — Clarifying Meaning 

In the final subsection, participants were asked to evaluate the importance of clarifying 
meaning. College graduates should have the ability to paraphrase or make clear the meaning of 
words, issues, conclusions, or beliefs (Halpem, 1992; Paul & Nosich, 1991). This skill 
includes the removal of confusing or ambiguous language (Facione, 1990). In order to achieve 
this goal, students need to identify and seek out specific resources to aid in obtaining the 
necessary information (Marzano et al., 1988). After the first round of surveys, the participants 
agreed that four skills were extremely important (see Table 36a). College graduates should be 
able to recognize confusing, vague language that requires clarification to increase 
comprehension; ask relevant and penetrating questions to clarify facts, concepts, and 
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Table 36b. Disagreements about Clarifying Meaning 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Recognize contradictions and 
inconsistencies in written or verbal 
language, data, images, or symbols 
Standard Deviation 


2.21 
1.12 


1.83 
1.03 


2.31 
1.19 


n.s. 


n.s. 


.0367 


Provide an example that helps to 
explain something or removes a 
troublesome ambiguity 

Standard Deviation 


2.81 
1.17 


2.23 
1.13 


2.44 
1.19 


.0074 


n.s. 


n.s. 


SKILLS — ROUND 2 


Recognize contradictions and 
inconsistencies in written or verbal 
language, data, images, or symbols 
Standard Deviation 


2.16 
.68 


1.80 
.70 


2.00 
.84 


.0147 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker: n.s. = not significant 



relationships; identify and seek additional resources, such as resources in print, that can help 
clarify communication; and develop analogies and other forms of comparisons to clarify 
meaning. After the second survey, a consensus was reached about one additional item that was 
rated as extremely important. College graduates should be able to provide an example that 
helps to explain something or remove a troublesome ambiguity. 

Faculty, employers, and policymakers disagreed about the importance of one item — should 
college graduates be able to recognize contradictions or inconsistencies in written or verbal 
language, data, images, or symbols? Faculty viewed this skill as extremely important and rated 
it si^^nificantly higher than employers did (see Table 36b). Faculty state,' ''Inconsistency/ 
contradiction indicates unclarity; to think critically, one must think clearly,'' and *'If one cannot 
recognize contradictions, then he or she can't reason.'' Policymakers tend to agree. They 
remark, "With the plethora of data and noise, the ability to take apart seemingly logical 
arguments and to bring clarity to masses of data [is] critical," and "Graduates must be 
discriminating readers." This skill is "of more importance to more mature graduates beyond 
the associate degree" remarks another policymaker. However, some faculty view this as a basic 
skill that is fundamental to successful critical thinkers. Employers did not note reasons for 
their differences in perspectives. 
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Table 37a- Analysis of Variance — Analysis Skills — Exaniining Ideas and Purpose 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Recognize the relationship between the purpose(s) of a 
communication and the problems or issues that must be 
resolved in achieving that purpose 


2.52 


2.10 


1.84 


.3204 


Assess the constraints on the practical applications of an idea 


2.89 


1.97 


1.91 


.5359 


Identify the ideas presented and assess the interests, 
attitudes, or views contained in those ideas 


2.14 


5.66 


1.50 


.0248* 


Identify the stated, implied, or undeclared purpose(s) 
of a communication 


2.35 


7.68 


1.60 


.0092* 


SKILLS — ROUND 2 


Identify the ideas presented and assess the interests, 
attitudes, or views contained in those ideas 


2.02 


1.04 


1.10 


.3910 


Identify the stated, implied, or undeclared purpose(s) 
of a communication 


2.00 


2.13 


.80 


.0718* 


* Significant differences noted inTUKEY and Least Square Means 



Cii. Analysis Skills 

Most scholars believe that the ability to identify the explicit and implicit features of a 
communication, especially in arguments that put forth a conclusion are critical skills (Facione, 
1990), The term argument is used to describe any form of thinking in which reasons are 
offered in support of a conclusion. Students should be able to discuss ideas in a manner that 
explores supporting and opposing points of views and try to increase understanding instead of 
just winning the argument (Chaffee, 1990), 

C>ii«a, Analysis Skills — Examining Ideas and Purpose 

One major area of analysis is to examine ideas and purposes (Facione, 1990). Here the three 
stakeholder groups agreed that two skills were approaching medium importance (see Table 
37a), College graduates should be able to recognize the relationship between the purposes of a 
communication and the problems or issues that must be resolved in achieving those purposes 
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Table 37b, Disagreements about Examining Ideas and Purpose 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Identify the ideas presented and 
assess the interests, attitudes, or 
views contained in those ideas 

Standard Deviation 


2.60 
1.45 


2.01 
1.19 


2.00 
1.03 


.0223 


.0391 


n.s. 


Identify the stated, implied, or 
undeclared purpose(s) of a 
communication 

Standard Deviation 


2.86 
1.55 


2.15 
1.19 


2.39 
1.10 


.0098 


n.s. 


n.s. 


SKILLS — ROUND 2 


Identify the stated, implied, or 
undeclared purpose(s) of a 
communication 

Standard Deviation 


2.27 
.94 


1.97 
.94 


1.71 
.46 


n.s. 


.0056 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



and should assess the constraints on the practical applications of an idea. After the second 
round of surveys, one additional skill was agreed upon — college graduates should be able to 
identify the ideas presented and assess the interests, attitudes, or views contained in those 
ideas. This skill was rated as extremely important. 

The three groups disagreed about the importance of whether college graduates should be able 
to identify the stated, implied, or undeclared purpose(s) of a communication (see Table 37b). 
Policymakers rated this skill significantly higher than did employers. Very few statements were 
written to express the differences in participants' views. Faculty noted that this skill is difficult 
for college graduates to learn. This skill "requires students to be more analytical which is 
necessary today,'' and ^'Reading between lines of communication is very important." However, 
another faculty member believes that "stated purposes are extremely important while implied 
or undeclared [purposes are] less important." This skill is "necessary for reading content-not 
just words" comments an instructor. A policymaker states, "Understanding what a 
communication is trying to accomplish is key to all other evaluations and action 
consequences." An employer emphasizes that "health care has a language that implies more 
than is stated frequently." Another manager notes, "These areas are crucial. Communication is 
the center of our business." Some employers believe that this skill requires experience that new 
college graduates do not possess. 
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Table 38a. Analysis of Variance — Detecting and Analyzing Arguments 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Examine a communication and determine whether or not it 
expresses a reason(s) in .support of or in opposition to some 
conclusion, opinion, or point of view 


2.02 


12.90 


1.39 


.0001* 


Identify the main conclusion of an argument 


1.57 


9.20 


.84 


.0001* 


Determine if the conclusion is supported with reasons and 
identify those that are stated or implied 


1.72 


10.94 


1.03 


.0001* 


Identify the background information provided to explain 
reasons which support a conclusion 


2.41 


4.14 


1.38 


.0525* 


Identify the unstated assumptions of an argument 


2.433 


6.04 


1.72 


.0321* 


SKILLS — ROUND 2 


Identify the background information provided to explain 
reasons which support a conclusion 


2.63 


.50 


.65 


.4672 


Identify the unstated assumptions of an argument 


2.54 


3.22 


1.17 


.0676 


Examine a communication and determine whether or not 
it expresses a reason(s) in support of or in opposition to 
some conclusion, opinion, or point of view 


L89 


3.32 


.58 


.0042* 


Identity the main conclusion of an argument 


1.57 


1.81 


.34 


.0057* 


Determine if the conclusion is supported v/ith reasons and 
identify those that are stated or implied 


1.82 


4.00 


.69 


.0038* 


* Significant differences noted in TUKEY and Least Square Means 



C.ii.b. Analysis Skills — Detecting and Analyzing Arguments 

An equally important skill is detecting and analyzing arguments. After the first round of 
surveys, there was no agreement about the importance of these items (see Table 38a). After the 
second round of surveys, there was agreement in two areas: college graduates should be able 
to identify the background information provided to explain reasons which support a 
conclusion and should be able to identify the unstated assumptions of an argument. However, 
the respondents continued to disagree about the importance of three skills. 
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Faculty rated significantly higher than did employers the ability to identify the main 
conclusion of an argument and to determine if a conclusion is supported with stated or implied 
reasons (see Table 38b). In terms of identifying the main conclusion of an argument, faculty 
remarked that it is the "most significant task of education," "if one doesn't know the intended 
conclusion, one cannot assess an argument,'' and it is "essential to the process — otherwise 
[college graduates] miss the point and focus on details." Some faculty also believe that this is a 
foundation skill required for more sophisticated thinking. As one professor notes this is "the 
minimum expectation for a reader." However, an employer questions whether college graduates 
actually have the ability to use this skill ''because of the unsaid, the main conclusion could be 
obscure." Other faculty believe that identifying the main conclusion is basic and as a professor 
notes, it is "the minimum expectation for a critical reader." A policymaker emphasizes that this skill 
is extremely important "so that you make the correct decision for support of nonsupport." 

Some respondents stressed the necessity of determining if a conclusion is supported with 
reasons and identifying those reasons as stated or implied. Some faculty believe this skill is 
difficult for college graduates to attain while others remark that it must be done to validate a 
conclusion. "If one cannot identify the reasons, one cannot appraise an argument." A 
policymaker states, "So much in the media and in organizations is focused on selling a point of 
view rather than bringing together different views. Sound bites tend to be sloppily or misleadingly 
constructed, making this analytical skill increasingly important." Another policymaker stresses that 
this skill is important in order to "make sure one is not emoting thinking." 

Faculty rated significantly higher than did employers the importance of the ability to 
determine whether a communication expresses a reason(s) in support of or in opposition to 
some conclusion, opinion, or point of view (see Table 38b). The faculty comment, "This is an 
absolute prerequisite to any reasoning," and "This is essential for argument evaluation." A 
faculty member questions, "Isn't this the whole point of reasoning?" while another professor 
stresses, "To avoid *gut feeling' decision, one should look for substantiation. A lot of this is reading 
comprehension." Most employers and policymakers tended not to offer reasons for their positions. 

In this entire analysis section of the survey, the respondents agreed that five skills were 
important for college graduates to achieve. The ability to identify the ideas presented and 
assess the interests, attitudes or views contained in those ideas was rated most important followed 
by the ability to recognize the relationship between the purpose of a communication and the 
problem issues that must be resolved in achieving that purpose. The importance of identifying the 
unstated assumptions of an argument and the background information provided to explain reasons 
which support a conclusion were rated extremely important. The ability to assess the constraints on 
the practical applications of an idea was rated slightly lower in importance than these other skills. 



C.iii. Evaluation Skills 

The ability to assess the credibility of a communication and the strengths of claims and 
arguments is critical (Chaffee, 1990; Ennis, 1987; Facione, 1990; Halpem, 1992; Paul & 
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Table 38b. Disagreements about Detecting and Analyzing Arguments 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND I 


Examine a communication and 
determine whether or not it 
expresses a reason(s) in support of 
or in opposition to some conclusion, 
opinion^ or point of view 

Stn n flfi rd Dtivicit io n 


2.67 
1.36 


1.75 
1. 11 


2.08 
1.18 


.0002 


,0457 


n.s. 


Identify the main conclusion of an 
argument 

Sicindcjrd Davicitiofi 


2.14 
1.37 


1.37 
.75 


1.50 
.74 


.0011 


,0107 


n.s. 


Determine if the conclusion is 
supported with reasons and identify 
those that are stated or implied 

Standard Deviation 


2,26 
1.36 


1.45 
.81 


1.92 
1.10 


.0007 


n.s. 


.0239 


Identify the background information 
provided to explain reasons which 
support a conclusion 

Standard Deviation 


2.71 
1.33 


2.24 
I.IO 


2,58 
1.23 


.0426 


n.s. 


n.s. 


Identify the unstated assumptions 
of an argument 

Standard Deviation 


2.90 
1,51 


2.29 
I.3I 


2.31 
1.05 


.0232 


,0477 


n,s. 


SKILLS — ROUND 2 


Examine a communication and 
determine whether or not it 
expresses a reason(s) in support of 
or in opposition to some 
conclusion, opinion, or point of view 
Standard Deviation 


2.27 
.57 


1.75 
.56 


1.90 
.68 


.0049 


n.s. 


n.s. 


Identify the main concl':sion of an 
argument 

Standard Deviation 


1.85 
1.15 


1.46 
.72 


1.57 
.70 


,0014 


n.s. 


n.s. 


Determine if the conclusion is 
supported with reasons and identify 
those that are stated or implied 

Standard Deviation 


2.24 
1.15 


1.67 
.72 


1.76 
.70 


.0104 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = 


not significant 
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Nosich, 199 1). There were a number of skills in this area where an agreement was reached. 
The most important skill in this section was the ability to determine if an argument rests on 
false, biased, or doubtful assumptions. Next the ability to evaluate the credibility, accuracy, 
and reliability of sources of information was cited as extremely important. Closely following 
was the skill of assessing the importance of an argument and determining if it merits attention. 
The ability to evaluate an argument in terms of its reasonability and practicality was important, 
followed by the assessment of statistical information used as evidence to support an argument. 
The remaining skills were approaching medium importance and included determining how 
new data might lead to the further confirmation or questioning of a conclusion; determining if 
conclusions based on empirical observations were derived from a sufficiently large and 
representative sample; and determining and evaluating the strength of an analogy used to 
warrant a claim or conclusion (see Table 39a). 

The respondents disagreed about the importance of six different skills in this section even after 
the second round of surveys (see Table 39b). Faculty and policymakers rated significantly 
higher than employers the ability to determine if an argument makes sense. However, some 
respondents in all three groups stress the importance of this skill, A faculty member states, 
"Making sense in daily living is important," while another professor questions, "Isn't this most 
of the goal?'' In a similar manner, an employer notes, "The ability to subject arguments to 
close scrutiny through the formulation of further questions is critical." This skill is the "most 
important overarching description of the skill. This is BASIC and ESSENTIAL," according to 
another employer. 

Faculty rated the ability to assess bias, narrowness, and contradictions when they occur in the 
person's point of view as significantly more important than employers did (see Table 39b). 
There were very few comments written about this skill. One professor stresses that "students 
cannot be too strong in this area" while another instructor notes that this is "a minimum 
expectation of a college graduate." "How can one be a critical thinker without this ability," 
questions a professor. Another faculty member believes that this skill is difficult for 
undergraduates to achieve with little life or work experience. 

The importance of assessing the clarity and consistency of language, terminology, and 
concepts employed in an argument was rated significantly higher in importance by faculty 
members than by the other two groups of participants (See Table 39b). A faculty member 
states, "If this is not done, it cannot be known if the argument is sound or unsound." "Clarity 
and consistency are necessary so that one knows exactly what one is being asked to believe or 
accept," emphasizes another professor. An instructor stresses that "inconsistency can be a big 
problem due to shifting definitions." This skill is a key to "good assessment of critical 
thinking," according to another professor. However, some faculty believe that this skill is 
lacking in our college graduates. 

Some employers agree that assessing the clarity and consistency of language is very important. 
A manager states, "Concepts are our business as is clear communication." Another employer 
notes that college graduates "must be able to identify language and terminology and how it 
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Table 39a* Analysis of Variance — Evaluation Skills 





Mean 


Mean 
Square 

Model 


Mean 
Square 

Error 


Pr>F 


SKILLS — ROUND 1 


Assess the importance of an argument and determine if it 
merits attention 


2.37 


.62 


1.86 


.7165 


Evaluate an argument in terms of its reasonabilily ^ind 
practicality 


2.50 


1.22 


1.79 


.5057 


Evaluate the credibility, accuracy, and reliability of sources 
of information 


2.14 


2.75 


1.628 


.1878 


Determine if an argument rests on false, biased, or doubtful 
assumptions 


2.04 


2.02 


1.57 


.2794 


Assess statistical information used as evidence to support 
an argument 


2.51 


1.18 


2.05 


.5626 


Assess how well an argument anticipates possible 
objections and offers, when appropriate, alternative 
positions 


2.99 


.48 


2.31 


.8137 


Determine how new data might lead to the further 
confirmation or questioning of a conclusion 


2.63 


.53 


1.79 


.7457 


Determine and evaluate the strength of an analogy used to 
warrant a claim or conclusion 


2.92 


3.93 


2.03 


.1469 


Determine if conclusions based on empirical observations 
were derived from a sufficiently large and representative 
sample 


2.87 


3.60 


2.95 


.2972 


Determine if an argument makes sense 


1.94 


3.11 


1.40 


.1120* 


Assess bias, narrowness, and contradictions when they 
occur in the person's point of view 


2.23 


6.88 


1.69 


.0187* 


Assess the degree to which the language, terminology, 
and concepts employed in an argument are used in an 
clear, consistent manner 


2.64 


8.75 


1.91 


.0113* 


Determine what stated or unstated values or standards of 
conduct are upheld by an argument and assess their 
appropriateness to the given context 


2.88 


6.01 


2.43 


,0867* 


continued on next page 
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Table 39a. Continued 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


Judge the consistency of supporting reasons, 
including their relevancy to a conclusion and their 
adequacy to support a conclusion 


2.42 


5.43 


1.65 


.0396* 


Determine and judge the strength of an argument in which 
an event(s) is claimed to be the result of another event(s) 
(causal reasoning) 


2.51 


12.53 


2.01 


.0024* 


SKILLS — ROUND 2 


Determine if an argument makes sense 


1.73 


3.36 


.70 


.0098* 


Assess bias, narrowness, and contradictions when they 
occur in the person's point of view. 


2.01 


2.71 


.79 


.0358* 


Assess the degree to which the language, terminology, and 
concepts employed in an argument are used in an clear, 
consistent manner 


2.51 


5.04 


1.04 


.0091* 


Determine what stated or unstated values or standards of 
conduct are upheld by an argument and assess their 
appropriateness to the given context 


2.77 


3.69 


1.09 


.0367* 


Judge the consistency of supporting reasons, including 
their relevancy to a conclusion and their adequacy to 
support a conclusion 


2.36 


6.37 


.79 


.0005* 


Determine and judge the strength of an argument in which 
an event(s) is claimed to be the result of another event(s) 
(causal reasoning) 


2.46 


4.82 


1.03 


.0106* 


* Significant differences noted inTUKEY and Least Square Means 



relates." Several references are made to health ceire and how this skill is particularly important 
in this context, A faculty member states this skill "is veiy important in health care 
documentation/' An employer stresses, ''Health care evokes a ton of emotion; therefore, 
language can escalate or deteriorate very easily.*' The comments about health care illustrate the 
importance of the abilities of recent college graduates to assess consistent language in a policy 
area that directly affects them as citizens in our society. 

The determination of whether stated or unstated values or standards of conduct are upheld by 
an argument and their appropriateness to the given context was rated significantly higher by 
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Table 39b. Disagreements about Evaluation Skills 





Means 


Significance Level 


E.MP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Determine if iin argument miikes sense 
Standard Deviation 


2.26 
1.23 


1.81 
1.20 


1.97 
1.08 


.0458 


n.s. 


n.s. 


Assess bias, narrowness, and 
contradictions when they occur in 
the person's point of view 

Standard Deviation 


2.71 
1.50 


2.04 
1.30 


2.25 
1.00 


.0130 


n.s. 


n.s. 


Assess the degree to which the 
language, terminology, and concepts 
employed in an argument are used 
in an clear, consistent manner 

Standard Deviation 


3.02 
1.35 


2.38 
1.42 


2.97 
1.30 


.0117 


n.s. 


.0240 


Determine what stated or unstated 
values or standards of conduct are 
upheld by an argument and assess 
their appropriateness to the given 
context 

Standard Deviation 


3.33 
1.63 


2.80 
1.67 


2.61 
.99 


n.s. 


.0193 


n.s. 


Judge the consistency of supporting 
reasons, including their relevancy to 
a conclusion and their adequacy to 
support a conclusion 

Standard Deviation 


2.81 
1.38 


2.23 
1.24 


2.54 
1.31 


.0206 


n.s. 


n.s. 


Determine and judge the strength of 
argument in which an event(s) is 
claimed to be the result of another 
events(s) (causal reasoning) 

Standard Deviation 


3.10 
1.68 


2.22 
1.37 


2.71 
, 1.23 


.0037 


n.s. 


.0487 


SKILLS — ROUND 2 


Determine if an argument makes sense 2.12 1.64 1,52 .0261 .0102 n.s. 
Standard Deviation 1.11 .78 .51 


Assess bias, narrowness, and 
contradictions when they occur in 

the person's point of view 2.33 1.87 2.10 .0256 n.s. n.s. 
Standard Deviation 1.02 .92 .44 


continued on next page 
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Table 39b. Continued 





Means 


Significance Level 


EMP 


FAC 


PM 


E.V1P/FAC 


EMP/PM 


FAC/P.Vl 


Assess the degree to which the 
language, terminology, and concepts 
employed in an argument are used 
in an clear, consistent manner 

Cf/ffi /•//!»'// D/j\'}/iti/tn 
tjlU'lUUrU Ly*: \ lUlK/fi 


2.91 
1.33 


2.32 
.95 


2.76 
.70 


.0229 


n.s. 


.0187 


Determine what stated or unstated 
values or standards of conduct are 
upheld by an argument and assess 
their appropriateness to the given 
conic ai 

Standard Deviation 


3.18 
1.13 


2.63 
1.08 


2.76 
.70 


.0183 


n.s. 


n.s. 


Judge the consistency of supporting 
reasons, including their relevancy 
to a conclusion and their adequacy 
to support a conclusion 

Standard Deviation 


2.82 
.98 


2.14 
.83 


2.62 
.97 


.0009 


n.s. 


.0470 


Determine and judge the strength 
of argument in which an event(s) 
is claimed to be the result of another 
events(s) (causal reasoning) 

Standard Deviation 


2.94 
1.22 


2,32 
■ .97 


2.38 
.80 


.0110 


.0482 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



faculty than by employers. A professor states, ^'Knowing the context is extremely important.^' 
**My view is that arguments inevitably engage the dimension of values,'' notes another 
instructor. A policymaker tends to agree with these statements and emphasizes, '*With 
increasing contention and complexity in our world, values are what can bring us together and 
provide stability and a sense of purpose. It is important for college graduates as leaders to 
understand and develop their own values and use them to guide action and conduct actions that 
encourage them in others." 

Some employers did not view values as an important element to assess in arguments. They 
viewed their own organizations as dealing with fnctual information that did not necessitate an 
evaluation of values. As one employer notes, '^Business communications involve less arguing 
over values and conduct." Other employers believe this is an area of weakness for recent 
college graduates. 
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Faculty rated the ability to judge the consistency of supporting reasons, including their 
relevancy to a conclusion and their adequacy to support a conclusion, significantly higher than 
employers and policymakers did (see Table 39b). A faculty member notes, "Consistency and 
relevance in evidence (reasons) are important in deciding whether one ought to accept the 
conclusion of an argument " This skill is "essential for understanding or else cognitive 
dissonance occurs" stresses another instructor Many faculty view this skill as extremely 
important and central to the development of good thinkers. They believe that critical thinking 
as an entire concept needs to be emphasized more in undergraduate education rather than 
simply the rote memorization of facts. Policymakers also stress the importance of this skill. 
They state, "Tnis makes the difference between simple assertion and reasoned argument," 
"Consistency is important for a reasoned position," and "With the amount of communication 
that people have to deal with everyday, evaluative skills should be practiced so they're instinct 
can make a person lots more productive." Some policymakers assert that unless college 
graduates possess this skill they can not draw logical and sound conclusions without the ability 
to judge supporting reasons. 

The importance of determining and judging the strength of an argument in Vv'hich an event(s) is 
claimed to be the result of another event(s) was rated significantly higher in importance by 
faculty members and policymakers than by employers (see Table 39b). Overall, faculty believe 
that some familiarity with causality is essential and a fundamental skill. A professor 
emphasizes, "Causal connections (e.g., smoking and cancer) are among the most significant 
claims we have to assess, since they so often have behavioral implications." Some employers 
also expressed the importance of this skill. A manager states, "Business is constantly sorting 
out why things happened in order to control future outcomes." A lawyer stresses the need to 
"have strong logical reasoning skills to work in departments that have legal implications." A 
health care employer emphasizes this skill's importance within his own organizational context. 
He states that individuals must review incident reports and determine "how patients get a 
wrong medication. [There could be] 20 variables involved." There are some corporations 
where causal reasoning is very important. When problems arise, it is important for their 
employees to determine what creates the difficulties. 

In terms of evaluation skills, the respondents agreed about the importance of nine areas. The 
skill rated most important was the ability to determine if an argument rests on false, unbiased, 
or doubtful assumptions closely followed by the evaluation of the credibility, accuracy, and 
reliability of sources of information. The assessment of the importance of an argument and 
whether it warrants attention was rated extremely important, followed by the evaluation of an 
argument in terms of its reasonability and practicality. The assessment of statistical 
information used as evidence to support an argument was rated important. The remaining 
skills approached medium importance. They included the ability to determine how new data 
might lead to the further confirmation or questioning of a conclusion; to determine and 
evaluate the strength of an analogy used to warrant a claim or conclusion; and to assess how 
well an argument anticipates possible objections and offers, when appropriate, alternative 
positions. 



141 



Table 40. Analysis of Variance — Inference Skills — 
Collecting and Questioning Evidence 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND I 


Determine what is the most significant aspect of a problem 
or issue that needs to be addressed, prior to collecting evidence 


Z. 14 


.90 


1 .4o 


. j4 J / 


Formulate a plan for locating information to aid in 
determining if a given opinion is more or less reasonable 
than a competing opinion 


2.23 


.83 


1.24 


.5124 


Combine disparate pieces of information whose connection 
is not obvious, but when combined offers insight into a 
problem or issue 


2.66 


2.90 


1.85 


.2121 


Judge what background information would be useful to 
have when attempting to develop a persuasive argument 
in support of one's opinion 


2.39 


3.85 


1.20 


.0432 


Determine if one has sufficient evidence to form a 
conclusion 


1.89 


3.36 


1.21 


.0652 


* Significance differences noted in TUKEY and Least Square Means 



Civ. Inference Skills 

The ability to reason from what we know to form new knowledge, draw conclusions, solve 
problems, explain, decide and/or predict is important (Facione, 1990). 

C.iv.a. Inference Sk ills — Collecting and Questioning Evidence 

A sub-skill in this area is the skill of "querying evidence." This skill involves the collection 
and questioning of evidence. In this subsection, respondents agreed about the importance of all 
five critical thinking skills contained in this segment (see Table 40). College graduates should 
be able to determine the most significant aspect of a problem or issue that needs to be 
addressed, prior to collecting evidence. The formulation of a plan for locating information to 
aid in determining if a given opinion is more or less reasonable than a competing opinion is 
extremely important. Furthermore, the ability to combine disparate pieces of information 
whose connection is not obvious, but, when combined, offers insights into a problem or issue 
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was considered important. College graduates should also be able to determine if one has 
sufficient evidence to form a conclusion. The judgment of what background information 
would be useful to have when attempting to develop a persuasive argument in support of one's 
opinion is essential. 



C.iv.b. Inference Skills — Developing Alternatives and Hypotheses 

The development of alternatives and hypotheses are considered to be important skills (Facione, 
1990; Halpem, 1992). This ability includes the consideration of both pros and cons of each 
alternative when making decisions. In this subsection, the respondents agreed about the 
importance of four specific skills after the first round of surveys (see Table 41a). College 
graduates should be able to seek evidence to confirm or disconfirm alternatives; seek the 
opinion of others in identifying and considering alternatives; assess the risks and benefits of 
each alternative in deciding between them; and, after evaluating the alternatives generated, 
develop when appropriate a new alternative that combines the best qualities and avoids the 
disadvantages of previous alternatives. They disagreed about the importance of three skills 
after the first round of surveys (see Table 41b). 

Faculty rated significantly higher than did employers the ability to list alternatives and 
consider their pros and cons, including their plausibility and practicality, when making 
decisions or solving problems (see Table 41b). Faculty view this as an essential skill. Some 
professors, however, were troubled by the term "practicality'' and thought that this skill was 
not important in some contexts. Other faculty believed that college graduates would vary a 
great deal in their abilities to actually use this skill in different situations. A couple of 
professors viewed this skill as too advanced for community college graduates. Another 
instructor noted that the skill of listing alternatives and considering their pros and cons was 
only an intermediate step in critical thinking. He cautions that this particular skill is not the 
final or last step in the thinking process. Some employers were critical of college graduates' 
abilities to use these skills effectively. Since they did not believe college graduates possessed 
this skill, they gave lower ratings. 

The projection of alternative hypotheses regarding an event, and the development of a variety 
of different plans to achieve some goal was rated as significantly more important by the 
faculty and policymakers than by employers (see Table 41b). Faculty note that this item places 
an emphasis on imagination which is critical to generate alternatives. They believe this 
creativity fosters flexibility and breadth in a student's thinking. This type of skill also helps to 
*'avoid closed-mindedness'' according to another professor. While assessing the alternatives is 
a fairly routine task, the generation of alternatives is more important and more difficult for 
college graduates according to some faculty. A couple of faculty believed this skill was too 
difficult for community college graduates. A professor concludes, "If postsecondary education 
does not teach tliis skill, what does it provide?" Employers tended to rate this skill lower in 
part because they did not agree with the idea of generating different plans to achieve some 
goal. As one employer emphasizes, *in business, alternative hypotheses need to be resolved 
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Table 41a. Analysis of Variance — Inference Skills ~ Developing Alternative Hypotheses 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


ck:i1 r c ROUND I 










Seek the opinion of others in identifying and considering 
alternatives 


2.52 


1.88 


2.21 


.4290 


List alternatives and consider their pros and cons, including 
tneiT pidUSiDiiiiy ano praciicajuyt wnen iuokihj^ ucl-i^siuho 
or solving problems 


1.97 


4.58 


L23 


.0262* 


Project alternative hypotheses regarding an event, and 
develop a variety of different plans to achieve some goal 


2.49 


9.35 


1.72 


.0050* 


Recognize the need to isolate and control variables in order 
to make strong causal claims when testing hypotheses 


2.71 


5.77 


2.23 


.078 1* 


Seek evidence to confirm or disconfirm alternatives 


2.16 


1.74 


1.57 


.3338 


Assess the risks and benefits of each alternative in deciding 
between them 


2.41 


1.17 


2.06 


.5659 


After evaluating the alternatives generated, develop, when 
appropriate, a new alternative that combines the best 
qualities and avoids the disadvantages of previous 
alternatives 


2.54 


1.69 


1.69 


.3699 


SKILLS — ROUND 2 


List alternatives and consider their pros and cons, including 
their plausibility and practicality, when making decisions 
or solving problems 


L82 


.60 


.52 


.3204 


Project alternative hypotheses regarding an event, and 
develop a variety of different plans to achieve some goal 


2.44 


1.34 


.78 


.1856 


Recognize the need to isolate and control variables in 
order to make strong causal claims when testing hypotheses 


2.83 


1.31 


1.02 


.2782 


* Significant differences noted in TUKEY and Least Square Means 



into one clear direction to achieve a goal." According to some employers, it is most important 
to identify one plan to attain a particular goal. Although some employers disagree. A manager 
in health care notes, "Every patient has to have a plan of care and alternatives." For the 
employers, their particular responses may be more tied to their particular organizational 
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Table 41b. Disagreements about Developing Alternatives and Hypotheses 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


List alternatives and consider iheir 
pros and cons, including their 
plausibility and practicality, when 
making decisions or solving problems 
Standard Deviation 


2.36 
1.19 


1.81 
1.09 


2.03 
1.08 


.0118 


n.s. 


n.s. 


Project alternative hypotheses 
regarding an event, and develop a 
variety of different plans to achieve 
some goal 

Standard Deviation 


3.07 
1.40 


2.29 
1.27 


2.44 
1.32 


.0024 


.0457 


n.s. 


Recognize the need to isolate and 
control variables in order to make 
strong causal claims when testing 
hypotheses 

Standard Deviation 


3.14 
1.60 


2.53 
1.54 


2.75 
1.18 


.0362 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 



contexts. Those companies with fairly routine procedures such as accounting firms may have 
less need to generate multiple paths to some goal while in service-related companies such as 
hospitals, there is a need to generate different plans to achieve or maintain the health of 
patients. 

The need to isolate and control variables in order to make strong causal claims when testing 
hypotheses was rated significantly higher by faculty than by employers (see Table 41b). 
However, some faculty qualified their responses. They noted that this particular skill was more 
specific to certain disciplines and that not all college graduates should be expected to achieve 
this goal. Some faculty believed this skill was a most ''fundamental characteristic of scientific 
knowing'* and the *'only way to develop a scientific base of information." A professor remarks, 
"All of these relate to my background as a scientist and my concern that many students, for 
example, have trouble understanding graphs, and as a result can be easily swayed.'' These 
faculty believed this paiticular skill was more relevant for graduates seeking to enter scientific 
or research positions. A policymaker cautions that this skill requires more maturity than an 
associate degree candidate would possess. 
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upon the second round of surveys, participants reached an agreement about the importance of 
these three skills. College graduates should be able to consider pros and cons of alternatives; to 
project alternative hypotheses; and to isolate and control variables. 

C.iv.c. Inference Skills - Drawing Conclusions 

Another important component within this reasoning section is the ability of college graduates 
to draw conclusions (Facione, 1990). The ability to develop informed, well-reasoned 
conclusions which draw on the views of others but which represent an individual's own 
independent analysis/synthesis and conclusions is a critical skill (Chaffee, 1990). Respondents 
agreed that six specific skills in this area were important from the first round of surveys (see 
Table 42a). The highest-rated skill was the ability to seek various independent sources of 
evidence, rather than a single source of evidence, to provide support for a conclusion. The next 
important skill, rated extremely important, was to develop and use criteria for making 
judgments that are reliable, intellectually strong, and relevant to the situation at hand. This 
skill was followed by the ability to reason well with divergent points of view, especially with 
those with which one disagrees, in formulating an opinion on an issue or problem. Closely 
related was, analogies, brain storming, and trial and error. College graduates should be able to 
assess how the tendency to act in ways to generate results (that are consistent with one's 
expectations) could be responsible for experimental results and everyday observations. The 
application of appropriate statistical inference techniques to confirm or discomirm a 
hypothesis in experiments was an important skill. After the second round of surveys, 
participants agreed about the importance of the remaining two skills in this section. College 
graduates should be able to note uniformities or regularities in a given set of facts, and 
construct a generalization that would apply to all of these and similar instances. They should 
also employ graphs, diagrams, hierarchical trees, matrices, and models as solution aids. Through 
two rounds of surveys, the respondents reached a consensus regarding all skills in this sabsecuon. 



C.v. Presenting Arguments Skills 

Another essential area of critical thinking is in the presentation of argument skills. This 
involves clearly communicating and justifying the results of one's reasoning. After the first 
round of surveys, the respondents agreed about the importance of two skills (see Table 43a). 
College graduates should be able to present supporting reasons and evidence for their 
conclusion(s) which address the concerns of the audience. Furthermore, college graduates 
should be able to negotiate fairly and persuasively. Respondents disagreed about the 
importance of four skills. Upon the complerion of the second round of surveys, the three 
groups agreed that three additional skills were important. The ability to present an argument 
succinctly in such a way as to convey the crucial point of an issue was rated extremely 
important as was the ability to formulate accurately and consider alternative positions and 
opposing points of view, noting and evaluating evidence and key assumptions on both sides. 
College graduates also should cite relevant evidence and experiences to support their position. 
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Table 42a. Analysis of Variance — Inference Skills — Drawing Conclusions 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Enor 


Pr>F 


SKILLS — ROLNDl 


Assess how the tendency to act in ways to generate resuits 
that are consistent with one*s expectations could be 
responsible for experimental results and everyday observations 


3.10 


7.15 


3.22 


.1114 


Reason well with divergent points of view, especially with 
those with which one disagrees, in formulating an opinion 
on an issue or problem 


2.34 


2.08 


2.17 


.3855 


Develop and use criteria for making judgments that are 
reliable, intellectually strong, and relevant to the situation 
at hand 


2.15 


2.16 


1.81 


.3050 


Apply appropriate statistical inference techniques to 
confirm or disconfirm a hypothesis in experiments 


3.30 


1.20 


2.97 


.6697 


Use multiple strategies in solving problems including 
means-ends analysis, working backward, analogies, brain 
storming, and trial and enror 


2.66 


1.15 


2.77 


.6595 


Seek various independent sources of evidence, rather than 
a single source of evidence, to provide support for a conclusion 


2.05 


3.30 


1.40 


.0977 


Note uniformities or regularities in a given set of facts, and 
construct a generalization that would apply to all these and 
similar instances 


2.64 


10.35 


1.71 


.0028* 


Employ graphs, diagrams, hierarchical trees, matrices, and 
models as solution aids 


3.38 


8.81 


3.48 


.0821* 


SKILLS — ROLND 2 


Note uniformities or irregularities in a given set of facts, 
and construct a generalization that would apply to all of 
these and similar instances 


2.72 


.40 


.82 


.6120 


Employ graphs, diagrams, hierarchical trees, matrices, 
and models as solution aids 


3.05 


.64 


1.52 


.6545 


* Significant differences noted in TUKEY and Least Square Means 
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Table 42b. Disagreements about Drawing Conclusions; 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 




Note uniformities or irregularities in 
a given set of facts, and construct a 
generalization that would apply to 
all of these and similar instances 

Standard Deviation 


3.21 
1.59 


2.40 
1.17 


2.72 
1.36 


.0035 


n.s. 


n.s. 


Employ graphs, diagrams, 
hierarchical trees, matrices, and 
models as solution aids 

Standard Deviation 


3.24 
1.82 


3.61 
2.02 


2.83 
1.32 


n.s. 


n.s. 


.0089 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = 


not significant 







The one area where the respondents continued to disagree was the ability of college graduates 
to illustrate their central concepts with significant examples and show how these concepts and 
examples apply in real situations. Faculty rated this skill significantly higher than did 
employers (see Table 43b). There were few written comments to support individual positions. 
However, some faculty stress that this application skill requires higher level abilities from 
students than did some previous items in this survey. 



C.vi. Reflection Skills 

Reflection skills are considered to be necessary in order to monitor one's comprehension and 
correct one's process of thinking. Students should self-consciously monitor their own 
cognitive abilities, particularly by applying skills in analysis and evaluation to their inferential 
judgments with a goal towards questioning, confirming, validating, or correcting either one's 
reasoning or one's results (Facione, 19, . No consensus about the importance of these skills 
was reached from the first round. After the second round of surveys, respondents agreed about 
the importance of two skills (see Table 44a). College graduates should be able to make 
revisions in arguments and findings when self-examination reveals inadequacies. They should 
also apply the skills of their own analysis and evaluation to their arguments to confirm and/or 
correct their reasoning and results. Both of these skills were rated extremely important. 

Two of the groups disagreed about the importance of the ability to examine critically and 
evaluate vested interests, beliefs, and assumptions in supporting an argument or judgment. 
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Table 43a. Analysis of Variance — Presenting Arguments Skills 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Fr>F 


SKILLS — ROUND 1 


Present supporting reasons and evidence for their 
conclusion(s) which address the concerns of the audience 


1.91 


2.59 


1.69 


.2186 


Negotiate fairly and persuasively 


2.46 


.68 


2.43 


.7545 


Present an argument succinctly in such a way as to convey 
the crucial point of an issue 


1.72 


4.28 


1.35 


.0438* 


Cite relevant evidence and experiences to support their position 


2.04 


6.78 


1.30 


.0063* 


Formulate accurately and consider alternative positions and 
opposing points of view, noting and evaluating evidence 
and key assumptions on both sides 


2.17 


7.98 


1.56 


.0070* 


Illustrate their central concepts with significant examples 
and show how these concepts and examples apply in 
real situations 


2.32 


4.09 


1.71 


.0935* 


SKILLS — ROUND 2 


Present an argument succinctly in such a way as to convey 
the crucial point of an issue 


1.66 


.67 


.45 


.2299 


Cite relevant evidence and experiences to support their position 


1.75 


.81 


.39 


.1259 


Formulate accurately and consider alternative positions and 
opposing points of view, noting and evaluating evidence 
and key assumptions on both sides 


1.88 


.60 


.61 


.3713 


Illustrate their central concepts with significant examples 
and show how these concepts and examples apply in 
real situations 


2.04 


1.34 


.49 


.0676* 


* Significant differences noted in TUKEY and Least Square Means 



Faculty rated this skill significantly higher than did employers (see Table 44b). Many faculty 
stress the importance of this particular skill but note the difficulty of getting students to 
achieve it. A professor notes this is "very hard to do, [an] unrealistic expectation in most 
cases.'' Others question whether this can be taught or expected from the completion of an 
undergraduate education. "Being objective about oneself is critical for responsibility and 
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Table 43b. Disagreements about Presenting Arguments Skills 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Present an argument succinctly in 
such a way as to convey the crucial 
point of an issue 

Standard Deviation 


2.12 
1.50 


1.60 
.94 


1.63 
1.33 


.0418 


n.s. 


n.s. 


Cite relevant evidence and 
experiences to support their position 
Standard Deviation 


2.52 
1.38 


1.86 
.98 


2.03 
1.29 


.0059 


n.s. 


n.s. 


Formulate accurately and consider 
alternative positions and opposing 
points of view, noting and evaluating 
evidence and key assumptions on 
both sides 

Standard Deviation 


2.62 
1.27 


1.94 
1.21 


2.37 
1.35 


.0036 


n.s. 


n.s. 


illustrate their central concepts 
with significant examples and show 
how these concepts and examples 
apply in real situations 

Standard Deviation 


2.67 
1.44 


2.16 
1.28 


2.40 
1.22 


.0490 


n.s. 


n.s. 1 


SKILLS — ROUND 2 


Illustrate their central concepts with 
significant examples and show how 
these concepts and examples apply 
in real situations 

Standard Deviation 


2.27 
.76 


1.95 
.67 


2.10 
.70 


.0337 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = 



not significant 







leadership and not encouraged by the political atmosphere. Building this habit early is more 
feasible than hoping it will evolve with maturity." Some faculty comment on the importance of 
the elimination of personal biases so that individuals remain more adaptable to a variety of 
situations. However, employers disagree about the importance of this particular skill. When 
they compare it with other skills in the survey, they did not believe it was as important. 
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Table 44a. Analysis of Variance — Reflection Skills 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Apply the skills of own analysis and evaluation to their 
arguments to confirm and/or correct their reasoning and results 


2.15 


7.56 


1.77 


.0153* 


Critically examine and evaluate their vested interests, 
beliefs, and assumptions in supporting an argument or 
judgment 


2.09 


18.65 


1.66 


.0001* 


Make revisions in arguments and findings when 
self-examination reveals inadequacies 


1.88 


7.70 


1.33 


.0036* 


SKILLS — ROUND 2 


Apply the skills of own analysis and evaluation to their 
arguments to confirm and/or correct their reasoning and results 


2.05 


2.55 


.69 


.0268 


Make revisions in arguments and findings when 
self-examination reveals inadequacies 


1.76 


.05 


.40 


.8825 


Critically examine and evaluate their vested interests, 
beliefs, and assumptions in supporting an argument or 
judgment 


1.99 


2.29 


.91 


.0844* 


* Significant differences noteu m TUKEY and Least Square Means 



C.vii. Dispositions 

Dispositions are different from skills in that they are behavioral tendencies or traits of mind 
that concern how college graduates are inclined to use their thinking skills. After the first 
round of surveys, respondents agreed about the importance of nine tendencies (see Table 45a). 
The inclination rated the highest (extremely import -mt) was to be curious and inquire about 
how and why things work followed by flexibility and creativity in seeking solutions. The third 
most important trait was to exhibit honesty in facing up to one's prejudices, biases, or 
tendencies to consider a problem solely from one's own viewpoint. This was followed by the 
willingness to persevere and persist at a complex task. The disposition of being organized, 
orderly, and focused in inquiry or thinking was rated extremely important and was closely 
followed by the inclination to arrive at a reasonable decision in situations where there is more 
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Table 44b. Disagreements about Reflection Skills 





Means 


Significance Level 




EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Apply skills of analysis and 
evaluation to arguments to confirm 
and/or correct reasoning and results 
Standard Deviation 


2.67 
1.52 


1.96 
1.26 


2.11 
1.30 


.0098 


n.s. 


n.s. 


Critically examine and evaluate 
vested interests, beliefs, and 
assumptions in supporting an 
c*rgument or judgment 

Standard Deviation 


2.88 
1.56 


1.78 
1.18 


2.14 
L27 


.0001 


.0235 


n.s. 


Make revisions in arguments and 
findings when self-examination 
reveals inadequacies 

Standard Deviation 


2.31 
1.46 


1.65 
.99 


2.11 
1.24 


.0094 


n.s. 


.0491 


SKILLS — ROUND 2 


Critically e-camine and evaluate 
vested interests, beliefs, and 
assumptions in supporting an 
argument or judgment 

Standard Deviation 


2.30 
I.Ol 


1.87 
.99 


2.00 
.63 


.0383 


n.s. 


n.s. 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = 


not significant 







than one plausible solution. The behavioral tendency to find ways to collaborate with others to 
reach a consensus on a problem or issue was rated extremely important as was the ability to 
monitor one*s understanding of a situation and progress towards goals. The inclination to 
apply insights from cultures other than one's own was rated important. The respondents 
disagreed about the importance of five dispositions. 

However, after the second round of surveys, participants agreed that the remaining five 
dispositions were extremely important. College graduates should willingly self-correct and 
learn from errors made no matter who calls them to their attention; be fair-minded; seek truth 
and be impartial, even if findings may not support one's preconceived opinions; be open- 
minded and strive to understand and consider different points of view; value the application of 
reason and the use of evidence; and be intellectually careful and precise (see Table 45b). 
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Table 45a. Analysis of Variance — Disposition 





Meiui 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


SKILLS — ROUND 1 


Be curious and inquire about how and why things work 


L65 


.49 


.93 


.5923 


Be organized, orderly, and focused in inquiry or in thinking 


2.03 


,42 


1.35 


.7303 


Willingly persevere and persist at a complex task 


2.02 


1.59 


1.33 


.3044 


Be flexible and creative in seeking solutions 


1.75 


.73 


1.21 


.5463 


Be inclined to arrive at a reasonable decision in situations 
where there is more than one plausible solution 


2.05 


2.47 


1. 13 


.1154 


Apply insights from cultures other than their own 


2.42 


1.29 


2.63 


.6122 


Exhibit honesty in facing up to their prejudices, biases, or 
tendency to consider a problem solely from their viewpoint 


1.85 


2.12 


1.63 


.2757 


Monitor their understanding of a situation and progress 
toward goals 


2.21 


.84 


1.53 


.5807 


Find ways to collaborate with others to reach consensus on 
a problem or issue 


2.13 


.64 


2.02 


.7304 


Be intellectually careful and precise 


1.96 


4.93 


1.36 


.0288* 


Value the application of reason and the use of evidence 


1.70 


8.13 


1.15 


.0011* 


Be open-minded; strive to understand and consider 
divergent points of view 


1.43 


4.71 


.78 


.0028* 


Be fair-minded; seek truth and be impartial, even if the 
findings of an inquiry may not support one's preconceived 
opinions 


L46 


2.74 


.67 


.0186* 


Willingly self-correct and learn from errors made no 
matter who calls them to their attention 


1.66 


2.21 


.96 


.1029* 


SKILLS — ROUND 2 


Be intellectually careful and precise 


1.85 


.84 


.43 


.1441 


Value the application of reason and the use of evidence 


1.77 


1.30 


.49 


.0724 


continued on next page 
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Table 45a. Continued 





Mean 


Mean 
Square 
Model 


Mean 
Square 
Error 


Pr>F 


Be open-minded; strive to understand and consider 
divergent points of view 


1.65 


.25 


.72 


.7090 


Be fair-minded; seek truth and be impartial, even if the 
findings of an inquiry may not support one's preconceived 
opinions 


1.65 


.58 


.46 


.2872 


Willingly self-correct and learn from errors made no matter 
who calls them to their attention 


1.62 


.45 


.56 


.4460 


* Significant differences noted in TUKEY and Least Square Means 



Table 45b, Disagreements about Dispositioas 





Means 


Significance Level 


EMP 


FAC 


PM 


EMP/FAC 


EMP/PM 


FAC/PM 


SKILLS — ROUND 1 


Be intellectually careful and precise 
Standard Deviation 


2.38 
1.45 


1.81 
1.04 


1.94 
1.19 


.0237 


n.s. 


n.s. 


Value the application of reason and 
the use of evidence 

Standard Deviation 


2.19 
1.38 


1.48 
.81 


1.83 
1.36 


.0028 


n.s. 


n.s. 


Be open-minded; strive to 
understand and consider divergent 
points of view 

Standard Deviation 


1.79 
1.18 


1.26 
.66 


1.57 
1.06 


.0083 


n.s. 


n.s. 


Be fair-minded; seek truth and be 
impartial, even if the findings of 
an inquiry may not support one's 
preconceived opinions 

Standard Deviation 


1.74 
1.17 


1.33 
.65 


1.54 
.82 


.0359 


n.s. 


n.s. 


Willingly self-correct and leam 
from errors made no matter who 
calls them to their attention 

Standard Deviation 


1.71 
1.22 


1.54 

.90 


1.94 
.91 


n.s. 


n.s. 

i 


.0268 


EMP = Employer; FAC = Faculty; PM = Policymaker; n.s. = not significant 
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Table 46a. Critical Thinking Goals Survey — Factor Analysis — Factors 1 and 2 



Goals 


Factor 1 


Goals 


Factor 2 


Recognize the use of misleading 
language 


.78897 


Assess how expectations and behaviors 
can influence results 


.70364 


Detect the use of leading questions 
biased for a specitlc response 


.74989 


Combine disparate pieces of information 
for new insights 


.67512 


Recognize the use of slanted detlnitions 
or comparisons 


.72940 


Analyze and evaluate their own 
arguments and reasoning 


.63378 


Detect the use of strong emotional 
language or imagery 


.71477 


Project alternative hypotheses and 
develop a variety of strategies 


.59025 


Detect if, then" statements based 
on false assumptions 


.66868 


Recognize the need to isolate and 
control variables 


.58754 


Detect irrelevant topics or considerations 
that obscure the main issue 


.58304 


Judge consistency, relevance, and 
adequacy of supporting reasons 


.53850 


Assess bias, narrowness, and 
contradictions 


.56956 


Consider alternative positions and 
opposing points of view 


.49142 


Determine what values and standards are 
upheld in an argument 


.56059 


Determine and evaluate the strength of 
an analogy 


.48374 


Examine and evaluate vested interests, 
beliefs, and assumptions 


.47944 


Note uniformities and regularities and 
construct generalizations 


.45776 


Recognize confusing or ambiguous 
language that requires clarification 


.40 1 /D 


List alternatives and consider pros and 
cons 


.41465 


Identify the interests, attitudes, or views 
111 d picDciucu iciea 




Illustrate central concepts with 
significant examples 




Identify the unstated assumptions 


.40608 






Make revisions in arguments when 
inadequacies are revealed 


.40473 






Identify the stated, implied, or 
undeclared purpose(s) 


.40044 






Percent of variance accounted for by factor 


9.12 


Percent of variance accounted for by factor 


6.63 



C.viii. Factor Ana lysis 

The items in the critical thinking survey were further analyzed by conducting a principle- 
components factors analysis with varimax rotation. This procedure was used to test the validity 
of the proposed structure of the underlying variables and to gain further insights into these 
constructs. For the critical thinking area, 12 factors were extracted since the original 
conceptualization consisted of 12 dimensions that were believed to account for the critical 
thinking variables. The rotated factor matrix is illustrated in Tables 46a-46f. 
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Table 46b. Critical Thinking Goals Survey — Factor Analysis — Factors 3 and 4 



Goals 


Factor 3 


Goal 


Factor 4 


Seek evidence to confirm or discontlrm 
alternatives 


.67940 


Find ways to collaborate with others 
to reach consensus 


.81314 


Assess the risks and benefits of each 
alternative before choosing 


.66776 


Be flexible and creative in seeking 
solutions 


.70372 


Present supporting reasons and evidence 
for their conclusion(s) 


.58733 


Monitor understanding of a situation 
and progress toward goals 


.69209 


Cite relevant evidence and experiences 
to support their position 


.57757 


Apply insights from cultures other 
than their own 


.61964 


Negotiate fairly and persuasively 


.56621 


Face up to their prejudices, biases, 
and dogmatism 


.56707 


After evaluating alternatives, develop 
better alternatives 


.53760 


Arrive at a reasonable decision when 
there are several possibilities 


.43387 


Seek the opinion of others in identifying 
and considering alternatives 


.53249 


Recognize the relationship between 
purpose i\nd problems aiid issues 


.43151 


Present an argument succinctly to convey 
the crucial point of an issue 


.51953 


Be curioLS and inquire about how and 
why things work 


.35034 


Seek vaiious independent sources of 
evidence to provide support 


.36930 






Percent of variance accounted for by factor 


6.21 


Percent of variance accounted for by factor 


5.84 



Many of the indirect persuasion items loaded most heavily on Factor 1. These included the 
abilities to recognize the us'^ misleading language, leading questions that are biased towards 
eliciting a preferred respon lonted definirions or comparisons which express bias, strong 
emotional language or ima trigger a certain response, if-then statements based upon 

false assumptions, and irrelevant topics or considerations that divert attention from issues. 
Additional items in the areas of evaluating assumptions also loaded onto this factor. These 
included assessing bias, narrowness, and contradictions; determining stated or unstated values; 
critically examining vested interests, beliefs and assumptions; and identifying the unstated 
assumptions. This factor accounted for approximately nine percent of the vaiiance. 

Factor 2 consisted primarily of inference items. Many of these items dealt with the review of 
information to project alternative hypotheses. This factor accounted for approximately seven 
percent of the variance. 

Factor 3 was comprised of a mixture of items listed in the developing alternatives and 
hypotheses section and the presenting arguments segment. These items accounted for six 
percent of the variance. 
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Table 46c. Critical Thinking Goals Survey — Factor Analysis — Factors 5 and 6 



Goals 


Factor 5 


Goals 


Factor 6 


Employ graphs, diagrams, hierarchical 
trees, matrices, and models 


.77292 


Make comparisons; note similarities and 
differences 


.77637 


Apply appropriate statistical inference 
techniques 


.68996 


Formulate categories, distinctions, or 
frameworks to organize information 


.71769 


Assess statistical information used as 
evidence in an argument 


.64782 


Classify and group data, findings, and 
opinions 


.70696 


Use multiple strategies in solving 
problems 


.55195 


Translate information from one medium 
to another 


.64756 


Determine if conclusion is based on an 
adequate and representative sample 


.54094 


Recognize contradictions in language, 
data, images, or symbols 


.49486 


Determine how data might confirm or 
challenge aconclusion(s) 


.53483 


Ask relevant and penetrating questions 
to clarify information 


.42324 


Assess how well an argument anticipates 
and responds to objections 


.47815 






Determine and judge the strength of a 
causal reasoning argument 


.40647 






Percent of variance accounted for by factor 


5.74 


Percent of variance accounted for by factor 


5.17 



Table 46d. Critical Thinking Goals Sur\'ey — Factor Analysis — Factors 7 and 8 



Goals 


Factor 7 


Goal 


Factor 8 


Assess the importance and merit of an 
argument 


.64542 


Identify the main conclusion of an 
argument 


.74633 


Determine if an argument makes sense 


.54391 


Identify the stated or implied reasons 
supporting a conclusion 


.68017 


Assess the clarity and consistency of 
language and concepts 


.51091 


Determine if a reason supports or 
opposes an opinion or conclusion 


.63989 


Evaluate an argument in terms of 
reasonability and practicality 


.50878 


Identify background information 


.50735 


Determine if assumptions are false, 
biased, or doubtful 


.48524 






Evaluate the credibility, accuracy, and 
reliability of the source of information 


.45882 






Assess the constraints on the practical 
applications of an idea 


.31638 






Percent of variance accounted for by factor 


4.49 


Percent of variance accounted for by factor 


4.35 
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Table 46e. Critical Thinking Goals Survey — Factor Analysis — Factors 9 and 10 



Goals 


Factor 9 


Goals 


Factor 10 


Be fair-minded; seek truth and be 
impartial 


.66103 


Identify the most significant aspect of a 
problem or issue 


.66729 


Be open-minded; strive to understand 
and consider divergent views 


.66088 


Judge what background information 
would be useful to support an idea 


.66187 


Willingly self-correct and learn from 
errors 


.50088 


Formulate a plan for locating 
information to facilitate judgments 


.46153 


Value the application of reason and the 
use of evidence 




Reason well v/ith divergent points of 

V icw 


.36617 


Determine if one has sufficient evidence 
to form a conclusion 


.38854 






Develop reliable and relevant criteria for 
making judgments 


.36375 






Percent of variance accounted for by factor 


3.94 


Percent of variance accounted for by factor 


3.58 



Table 46f. Critical Thinking Goals Survey — Factor Analysis — Factors 11 and 12 



Goals 


Factor 1 1 


Goal 


Factor 12 


Be organized, orderly^ and focused in 
inquiry or in thinking 


.77545 


Provide an example to explain 
something or clarify an ambiguity 


.61140 


Be intellectually careful and precise 


.69031 


Identify and seek additional resources 
to clarify meaning 


.57574 


Willingly persevere and persist at a 
complex task 


.43593 


Develop analogies and other forms of 
comparison to clarify meaning 


.45942 


Percent of variance accounted for by 
factor 


2.96 


Percent of variance accounted for by 
factor 


2.57 



Factor 4 contained mainly dispositions items including the tendencies to find ways to 
collaborate, to be flexible and creative in seeking solutions, to monitor understanding of a 
situation, to apply insights from other cultures, to exhibit honesty, to arrive at a reasonable 
decision, and to be curious. This factor accounted for roughly six percent of the variance. 

Factor 5 comprised mainly skills related to statistical techniques, causal reasoning, and the 
evaluation of alternatives. For example, the ability to employ graphs and diagrams, to apply 
statistical 
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inference techniques to confirm or disconfirm hypotheses, and use multiple strategies 
including means-ends analysis were grouped together. This factor accounted for six percent of 
the variance. 

Factor 6 consisted of solely interpretation skills. The abilities to make comparisons, formulate 
categories, classify and group data, translate information from one medium to another, ask 
relevant questions to clarify facts, and recognize contradictions or inconsistencies accounted 
for approximately five percent of the variance. 

Many of the evaluation items loaded onto factor 7. These skills include the assessment of the 
importance of arguments and whether they warrant attention, if an argument makes sense, if an 
argument reflects reasonability and practicality, and an evaluation of the credibility, accuracy, 
and reliability of sources of information. This factor accounted for roughly four percent of the 
variance. 

Factor 8 was composed of items from the detecting and analyzing arguments section. These 
inclinations included the identification of the main conclusion of an argument, the 
determination of whether the conclusion is supported with stated or implied reasons, 
examination of whether a communication expresses reasons in support of opposition, and the 
identification of background information provided to explain reasons in support of a 
conclusion. This factor accounted for four percent of the variance. 

Factors 9 through 12 each accounted for less than four percent of the variance. Factor 9 
consisted of mainly four disposition items. Factor 10 was comprised of inference skills related 
to the collection and questioning of evidence. Factor 1 1 consisted of items relevant to the 
tendencies to be precise, organized, and persistent at a complex task. Factor 12 was comprised 
of skills related with clarifying meaning by providing examples, developing analogies, and 
seeking additional resources to explain something. 

This factor analysis indicates that many of the variables loaded onto a factor with other 
variables that they were originally grouped within the survey. However, the separate section 
for reflection skills was probably not necessary since most of these skills loaded onto Factor 1 
along with the other interpretation skills. In some sections of the original framework, there 
were items that were linked together that cut across areas. For example, indirect persuasion 
skills also included elements originally grouped in the evaluation section. These skills are 
interrelated and not clear, separate units with little connections. 



C.ix. Reliability 

The results from the reliability analysis for the critical thinking survey are presented in Table 
47. Within the interpretation section, the subsection of categorizing had the lowest reliability 
(fl= .84) while detecting indirect persuasion had the highest reliability (a= .93). In the analysis 
section, the reliabilities of both the examining ideas/purpose section and the detecting/ 

159 

ERLC 



Table 47. Reliability of Items in Critical Thinking Survey 



Sections 


Number of Items 


Alpha 


Interpretation 


16 


.9356 


Categorizing 


4 


.8367 


Detecting indirect persuasion 


6 


.9277 


\«vitiniying incdning 


0 


.0 /U / 




0 

y 




Examining ideas and purpose 


4 


.8620 


Detecting and analyzing arguments 


5 


.8775 


Evaluation 


15 


.9554 


Inference 


20 


.9462 


Collecting and questioning evidence 


5 


.8460 


Developing Alternatives and hypotheses 


7 


.8871 


Drawing conclusions 


8 


.8989 


Presenting arguments 


6 


.9030 


Reflection 


3 


.8946 


Dispositions 


14 


.9173 


Total criti al thinking survey 


83 


.9844 



analyzing arguments section were fairly similar (a= .86 and a= .88, respectively). In the 
inference section, the collecting and questioning evidence sub-unit had the lowest reliability 
{a= .85) while the drawing conclusions sub-unit had the highest reliability {a= .90). The 
presenting arguments, reflection, and disposition sections all had similar reliabilities. The 
evaluation section had the highest reliability {a= .95) when compared with all sections while 
the reflection section had the lowest reliability (a= .90). Overall, the reliability of the 
individual sections and subsections tended to increase with the number of items comprising a 
given unit. The majority of reliability coefficients were above a= .84. 

C.x. Summary 

Critical thinking is an educational value and is often reflected as a goal for many general 
education programs at colleges and universities. For example, the state universities in 
California require a course in critical thinking prior to graduation. Institutions with 
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competency-based curricular programs, such as Alvemo and King's Colleges, have integrated 
critical thinking programs at the core of their curriculum. Critical thinking is also an important 
national education goal. 

The purpose of the Delphi study presented in this report was to define the ideal skills that 
college graduates should possess to be effective citizens in society and employees in the 
workplace. The participants in this project reached a consensus about most elements that are 
considered important for good thinkers. Faculty, employers, and policymakers believe that 
certain interpretation skills are important. College graduates should be able to detect indirect 
persuasion including the use of leading questions that are biased towards eliciting a preferred 
response, the use of misleading language that exaggerates or downplays the importance of 
something, the use of slanted definitions or comparisons which express a bias for or against a 
position, and detect instances where irrelevant topics or considerations are brought into an 
argument to divert attention from the original issue. 

A key component of interpretation is the ability to categorize information. There were no 
disagreements in this area. The respondents agreed that college graduates should be able to 
make comparisons, formulate frameworks or categories, classify data, and translate 
information from one medium to another. Faculty, employers, and policymakers note that an 
equally important interpretation skill is the ability to clarify meaning. In oruer to make clear 
the meaning of words, issues, conclusions or beliefs, college graduates need to recognize 
confusing, vague language; ask relevant or penetrating questions; identify and seek additional 
resources; and develop analogies or other forms of comparisons; and provide examples to 
explain ideas. These results are consistent with the outcomes from Facione's (1990) Delphi 
study. 

In general, when there were disagreements between the groups about the importance of skills, 
faculty tended to rate them significantly higher than employers and policymakers. Most 
individuals considered these critical thinking skills to be important but what differed was the 
level of importance they gave to individual skills. For example, faculty believe that the ability 
to detect strong emotional language is critical. When employers and policymakers compared 
this skill to others on this list, they did not believe it was as important as the faculty rated it. 
Often the employers or policymakers did not find particular skills to be important within their 
own organizational contexts. Given the nature of the job responsibilities in certain companies, 
employers note that some skills are not neariy as important as others. In terms of detecting 
emotional language, the individuals with disagreements tend to stress that their organizations 
deal with objective, factual information where this type of language is not encountered in their 
view. However, policymakers did rate significantly higher than did both faculty and employers 
the importance of detecting "if, then'' statements. 

The ability to idenfify the explicit and implicit features of a communication, especially in 
arguments that put forth conclusions are essential skills according to the respondents. This 
supports the findings from Facione's (1990) study. Students should be able to examine ideas 
and purposes by assessing the constraints on the practical applications and by assessing the 
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interests, attitudes, or views contained in those ideas. The ability to identify stated, implied or 
undeclared purpose(s) of a communication was rated significantly higher by the policymakers. 
Some faculty noted the difficulties of teaching students this particular skill, and this perception 
could account for their lower ratings. However, faculty valued the ability to identify the main 
conclusion of an argument and to determine if a conclusion is supported with stated or implied 
reasons more than the employers or policymakers did. Funhermore, faculty rated higher than 
did these two groups the ability to determine whether a communication expresses a reason(s) 
in support of or in opposition to some conclusion or point of view. 

College graduates should be able to assess the credibility of a communication and evaluate the 
strengths of claims and arguments. In this area, there was a high degree of consensus among 
the participants. These results are consistent with the assertions made by a number of scholars 
(Chaffee, 1990; Ennis, 1987; Facione, 1990; Halpern, 1992; and Paul & Nosich, 1991). 
Specifically, students need to: determine if arguments rest on false, biased, or doubtful 
assumptions; evaluate the credibility, accuracy, and reliability of sources of information; assess 
the importance of an argument and determine if it merits attention; evaluate an argument in 
terms of reasonability and practicality; assess statistical information; determine how new data 
may lead to further confirmation or questioning of a conclusion; determine if conclusions are 
derived from sufficiently large and representative samples; and evaluate analogies. 

Faculty rated the ability to assess bias, narrowness, and contradictions as significantly more 
important than did employers. A similar pattern emerged in the areas of assessing the clarity 
and consistency of knowledge and determining whether stated or unstated values or standards 
of conduct are upheld by an argument. Faculty tended to view these skills as essential to the 
development of good critical thinkers. However, some employers did not believe that college 
graduates possess these skills. When they perceived college graduates as lacking certain skills, 
they tended to give them lower ratings. Some faculty also noted the difficulties of teaching 
students these particular skills. 

There were two additional areas where faculty rated skills higher than did both employers and 
policymakers. These areas included causal reasoning abilities and the judgment of consistency 
of supporting reasons. In all three groups, there were individuals who rated these skills as 
extremely important. However, due to the nature of certain companies, some employers and 
policymakers viewed these skills as less important. 

Faculty, employers, and policymakers agreed about the importance of all inference skills 
identified in the sur\'ey (see Table 48). The ability to collect and questioti evidence was an 
inference skill rated important by most participants. This skill involves the formuladon of a 
plan for locating information, the combination of disparate pieces of information, 
determination of sufficient evidence to form a conclusion, and the judgment of what 
background information would be useful. Equally important was the ability to develop 
alternatives and hypotheses. Respondents agreed that college graduates should be able to seek 
evidence to confirm or disconfirm alternatives; seek opinions of others; assess the risks and 
benefits of each option; and develop new alternatives when appropriate. The importance of 
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Table 48* Summary of Consensus and Disagreements in 
Each Section of Critical Thinking Survey 







Round 1 — 






Round 2 — 






Final — 




Section of Survey 


Number of Items 


Number of Items 


Number of Items 




A 


% 


D 


% 


A 


% 


D 


% 


A 


% 


D 


% 


In.erprelalion Skills 


8 


50.0 


8 


50.0 


5 


62.5 


3 


37.5 


13 


81.3 


3 


18.7 




4 


100.0 


0 


00.0 


— 


— 


4 


100.0 


0 


00.0 






Detecting Indirect 


























Persuasion 


0 


00.0 


6 


100.0 


4 


66.7 


2 


333 


4 


66.7 


2 


33.3 


Clarifying Meaning 


4 


66.7 


2 


33.3 


1 


50.0 


I 


50.0 


5 


83.3 


1 


16.7 


Analysis Skills 


2 


22.2 


7 


77.3 


3 


42.9 


4 


57.1 


5 


55.6 


4 


44.4 


Examining Ideas and 


























Purpose 


2 


50.0 


2 


50.0 


1 


50.0 


1 


50.0 


3 


75.0 


1 


25.0 


Detecting and Analyzing 


























Arguments 


0 


00.0 


5 


100.0 


2 


40,0 


3 


60.0 


2 


40.0 


3 


60.0 


Evaluation Skills 


9 


60.0 


6 


40.0 


0 


00.0 


6 


100.0 


9 


60.0 


6 


40.0 


Inference Skills 


15 


75.0 


5 


25.0 


5 


100.0 


0 


00.0 


20 


100.0 


0 


00.0 


Collecting and Questioning 


























Evidence 


5 


100.0 


0 


00.0 






5 


100.0 


0 


00.0 






Developing Alternatives 


























ajnd Hypotheses 


4 


57.1 


3 


42.9 


3 


100.0 


0 


00.0 


7 


100.0 


0 


00.0 


Drawing Conclusions 


6 


75.0 


2 


25.0 


2 


100.0 


0 


00.0 


8 


100.0 


0 


00.0 


Presenting Argument Skills 


2 


33.3 


4 


66.7 


3 


75.00 


1 


25.0 


5 


83.3 


1 


16.7 


Reflection Skills 


0 


00.0 


3 


100.0 


2 


66.7 


1 


33.3 


2 


66.7 


1 


33.3 


Dispositions 


9 


64.3 


5 


35.7 


5 


100.0 


0 


00.0 


14 


100.0 


0 


00.0 


TOTAL 


45 


54.2 


38 


45.8 


23 


60.5 


15 


39.5 


68 


81.9 


15 


18.1 



A = Number of Items for which there was agreement; D = Number of Items for which there was 
disagreement 



considering the pros and cons of alternatives when making decisions is consistent with the 
work done by Facione (1990) and Halpem (1992). The participants also agreed that the ability 
to draw conclusions was important. College graduates should be able to develop informed, 
well-reasoned conclusions which draw on the views of others but which represent an 
individual's own independent analysis/synthesis and their own summaries. Faculty, employers, 
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and policymakers faculty agreed that every item related to inference was important for college 
graduates to achieve. 

Another important skill is the ability to present arguments. This skill involves clearly 
communicating and justifying the results of one's reasoning. According to the respondents, 
college graduates should be able to present supporting reasons and evidence for their 
conclusions, present the crucial point of an issue, evaluate the key assumptions, and formulate 
accurately alternative positions. The illustration of central concepts with significant examples 
and how they apply to real situations was considered to be extremely important by the faculty 
members. However, some faculty and employers questioned whether college graduates should 
actually be expected to achieve this goal. They noted that this skill requires advanced abilities 
that some college graduates do not possess. 

Reflection skills are considered to be necessary in order to monitor one's comprehension and 
correct one's process of thinking. College graduates should be able to make revisions in their 
arguments when their own self-examination reveals inadequacies. They should also apply the 
skills of their own analysis to their own arguments to confirm and/or correct their reasoning 
and results. However, faculty rated the ability to critically evaluate interests, assumptions, and 
beliefs in supporting an argument significantly higher than did employers. Faculty believed 
these are essential skills while some employers viewed these as less important within their 
own organizations. 

Dispositions are behavioral tendencies or traits of mind that concern how college graduates are 
inclined to use their thinking skills. They agreed about the importance of all di'jpo^^'tions 
included in this survey. College students should be curious, organized, fair-mindev., open- 
minded, flexible, creative, and should persevere, apply insights from other cultures, find ways 
to collaborate, value the application of reason, and willingly self-correct and learn from errors. 

The participants in this study reached a consensus about many important critical thinking skills 
that college graduates should achieve upon the completion of their degree programs. These 
results reflect the perspectives of this group of participants. We make no assertion that the 
critical thinking skills agreed upon in this study are important for every college and university 
graduate. The participants outlined the specific ideals they considered to be crucial for college 
graduates to possess for citizenship and employment. These results provide information that 
others can consider and critique, and can use to determine applications within their own 
settings. These ideals may serve as considemtions for further dialogue and for a future goal of. 
guiding assessment and curricular reforms. 
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V. CONCLUSION: IMPLICATIONS FOR TEACfflNG AND LEARNING 



This study has focused upon the ideal skills that faculty, employers, and policymakers believe 
that college graduates should achieve. There is a cleai' consensus regarding the importance of 
certain basic as well as advanced skills. However, we do not know how well our college 
graduates are attaining advanced abilities within the areas of communication and critical 
thinking skills. Many focus group and advisory board members for this project stressed the 
need to link the results from this project with methods to improve instructional strategies and 
assessments of student learning within the classroom. These individuals believed that this 
work should be expanded and built upon in order to have an impact and bring about 
constructive improvements in higher education. In this section, we outline the implications of 
the findings, challenges, and future actions that may be developed to strengthen the 
undergraduate curriculum and enhance student learning. 

A. Linking Results with Instructional Activities in the Classroom 

A challenge facing higher education is how to increase the awareness and consciousness of 
faculty teaching in the classroom so that the development of students' communication and 
critical thinking skills are raised to a higher level. The traditional model of teaching in the 
classroom often consists of the professor lecturing while students take notes, read textbooks, 
and memorize material. Students in this classroom are then frequently assessed by 
examinations where they are asked to recall basic facts, terms or concepts. In this environment, 
students are passive learners. Faculty do the thinking for students and transmit information. 

A major factor that affects instructors' abilities to develop students' advanced skills is an 
overemphasis with having "students memorize the accepted answers and not be concerned 
enough with guiding them in the processes and methods of arriving at well founded answers" 
(Glaser, 1985, p. 25). Another reason for the lack of attention to promoting advanced skills is 
that many professors do not know how to teach these skills because *'they have never been 
provided with pedagogical methods for doing so" (King, 1994). Faculty members tend to teach 
in the same way that they were taught in their own undergraduate and graduate education. 
Despite these challenges, some educators have created materials and procedures or processes 
to guide students in the development of advanced skills in communication and critical thinking 
(for example, Angelo & Cross, 1993; Halpern, 1994; White, 1994). These skills can be 
developed by instruction designed specifically to enhance students' abilities (Facione, 1992; 
Halpern, 1992, 1994; and Paul, 1992). The empirical literature also confirms that cognitive 
development is associated with specific kinds of classroom activities and instructor behaviors 
(Ewell, 1994). 

Students must become more engaged with their own learning in order to help them develop 
their advanced skills. Faculty play a key role in guiding students io reach these higher levels of 
learning. Professors can use instructional techniques that stimulate students to pose questions 
and think of their own answers. The effective use of questions can help students to constmct 
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knowledge (King, 1994). There are a variety of instructional strategies under the heading of 
"active learning" that show promise for moving students to higher levels of learning. These 
strategies move students away from simply rewriting what they have learned from lectures or 
restating others' ideas. The puipose of active learning is to guide students to explore their own 
thinking about concepts or issues in order to expand their own mental processes and structures 
(Meyers & Jones, 1993). 

One major aspect associated with active learning is that students are given many opportunities 
to write, speak, and think critically across the curriculum rather than relying on the 
development of these <;kills solely in a single course. Faculty design class activities that give 
students the chance to practice these skills. For example, students should be given 
opportunities to write multiple drafts when the assignment is initially given and encouraged to 
make revisions until the assignment is turned in at the end of the term (White, 1994). Students 
can be encouraged to discover their own topics for development rather than being given a 
specific idea by the professor. Peer interaction in teams and other collaborative efforts are 
additional important aspects of active learning. There is a growing body of information about 
active learning strategies that may help students to improve their skills. 

B. Using the Goals Inventories 

One potential way to use the results from this project and the actual goals inventories is to 
begin or faciUtate discussions on campuses with groups of faculty about what their course 
goals, objectives, and outcomes are relative to communication skills and critical tliinking. The 
inventories developed for this project can serve as the broad array of considerations for goals 
in individual courses. In groups, faculty can rate the importance of these goals for the courses 
that they teach. If groups of faculty teach the same course at a particular college, they may 
wish to reach a consensus about the importance of certain skills. Faculty discussions about 
goals and outcomes relative to communication and critical thinking skills can help them to 
make informed judgements about what should be included in courses and how to structure 
classes so that students reach higher levels of learning. Once goals and outcomes are clearly 
articulated, then all faculty can work together to build these skills not only in general 
education courses but also in the more advanced courses that are required for the academic 
majors. In a similar manner, internship supervisors at the institution and within business 
organizations could evaluate how important these goals are for effective performance v/ithin 
their company and assess how well students are meeting these particular goals. This 
collaboration could help employers identify individuals who would be matched to achieve the 
demands of particular positions. 

Many experts and educators beheve that advanced skills such as critical thinking consist of 
general principles of reasoning that bridge subjects and have application to many subjects 
(Ennis, 1987; Facione, 1990). Halpem (1993) asserts that courses specifically designed to 
develop generic reasoning skills use diverse examples to provide the best practice with 
transferring these skills. Assessments of student learning should be based upon simulated 
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cases that are similar to contexts that students are exposed to outside of the classroom 
Effective assessments of communication and critical thinking in the classroom or at the 
program level would help faculty to determine if students are improving their abilities if there 
are weaknesses. In the areas of gains in student learning, faculty would be able to determine 
what particular instructional strategies or course activities are linked with student growth. The 
goal of assessment should be to identify which specific educational experiences or which 
practices result in gains in communication and critical thinking skills. 

In order to determine how well college graduates are achieving these goals nationally, Ewell 
(1994) has outlined a vision statement for the usefulness of 'instructional good practice 
indicators in undergraduate education.'' He concludes that the evidence from an extensive 
literature review reveals that there are much stronger connections between desired outcomes 
and what happens in classes and what students actually do than can be gained from a particular 
institution's curricular structure. This statement supports the calls for action by the focus 
groups and advisory boards of this project. Ewell suggests that the best method to determine 
indicators of good practice is to survey national samples of faculty and administrators with 
questionnaires to gain information about faculty-student contact and specific student behaviors 
related to actual time on certain tasks as well as their quality of effort. While these activities 
may present some useful beginning information, they are primarily highlighting faculty ai'.d 
student perceptions mther than their actual behaviors. 

Faculty often state that they are seeking to develop students' abilities to analyze, synthesize, 
and think critically. However, research indicates that faculty do not follow their good 
intentions when they develop their courses. A formal review and analysis of course syllabi and 
exams revealed that college faculty do not in reality focus on these advanced skills md instead 
are far more concerned with students' abilities to acquire knowledge, comprehend basic 
concepts or ideas and terms, and apply this basic knowledge (Ratcliff, 1994; Ratcliff, Jones, 
Guthrie, & Oehler, 1991). While gathering data on actual behaviors is more difficult, it is 
necessary since perceptions do not match actions. 

A formal and systematic review of cross-sectional samples of collegiate assignments and 
examinations by the faculty members who teach courses could provide useful information. 
Expectations and levels of advanced achievements could be more clearly defined. In order to 
determine if college students have become better critical thinkers or communicators, an 
evaluation would need to be tied to the goals that were specifically targeted in the courses. 
Innovative, model course c ctivities and assessment techniques that faculty have designed to 
develop students' communication and critical thinking skills could be identified and serve as 
examples that other faculty who strive to improve their own courses could consider. 

The purpose of this study was not to provide a single definition of important communication 
and critical thinking skills rather this work is intended to stim.ulate discussion from these 
extensive lists of goals so that faculty may adapt, modify, or decide which goals may be most 
appropriate for their own college students. A strength of modem colleges and universities is 
that they provide a universe of knowledge and curriculum that is the mirror of a 
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technologically, socially, and economically complex society. Such cunicular complexity has 
been a necessity given the diversity of students who enroll in postsecondaTy programs with 
different goals, interests, and expectations. A postsecondary assessment must reflect and 
describe the diversity of that curriculum if it is to enhance our students' abilities. 

The project outlined in this report begins a formal investigation into the skills that faculty, 
employers, and policymakers believe are critical for college graduates to attain. The findings 
are based upon the perspectives of the participants in this study. While these individuals do not 
constitute a representative sample, they do move us towards a better understanding of what 
skills are considered important and define these skills in greater specificity. 

This study has focused on primarily learning within the formal environment of courses and 
academic programs. However, there is a great deal of literature that documents the 
contributions of experiences outside of the classroom that may influence the development of 
communication and critical thinking skills (Pascarella and Terenzini, 1991). The development 
of these skills does not end with a college degree. Once students are hired into their 
professional jobs, they become members of a new culture. Individual companies have different 
cultures which influence what is and is not acceptable in communications with other 
employees. For example, individual companies develop their own formats for written 
documents and these formats vary considerably from one company to another. The effective 
writing abilities of college graduates are in part influenced by their ability to learn the local 
norms and standards of good practice which govern the conventions of quality written 
documents within a particular organizational culture. Employees often receive additional 
training once they enter the workplace. 

Ideally an effective undergraduate education helps students become better thinkers, 
communicators, and decision makers in the real world contexts of work and society that 
extend beyond the traditional classroom. Halpem (1993, p. 242) asserts that "ideally, the 
students who hav^. become better thinkers will demonstrate critical thinking skills that range 
from more reasoned consumerism to improved problem solving.'' 

Students need to acquire basic communication skills and develop cognitive abilities to 
understand principles, concepts or ideas. However, students must move beyond being simply 
receivers or transmitters of information. In order to reach advanced skills in writing and speech 
as well as listening, college students need to develop their critical thinking skills in order to 
evaluate, analyze, and make judgements about the multitude of messages or interactions they 
encounter in their daily lives. Ideally, college graduates will learn to assume responsibility for 
their own intellectual development that will continue beyond the formal education they receive 
in college. The improvement of these skills should help students to become better citizens and 
employees in real world contexts. 
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